AI Companies Are Building for the Wrong Users

Why building for your biggest fans might not work forever

A close friend is a software engineer at one of the most established technology companies in the world — a perennial member of whatever your favorite big tech acronym is. They’ve been there for close to a decade, and recently, they’ve been working to boost their own productivity using an internal AI coding agent. As they’ve shared what’s working and where the limitations of the agent are within their team, interest quickly spread in their broader org. They ended up giving a talk to hundreds of engineers inside the company.

Source: Gemini.

We were fascinated to hear about their experience. Despite being an engineer at one of the most technically sophisticated companies in the world – one that builds its own foundation models! – they encountered significant and widespread reluctance to adopt coding agents. Some engineers simply liked writing code by hand — it was part of how they thought about their craft. Others couldn’t figure out how the tool was supposed to fit into their workflow. Still others had tried it, found the UX confusing, and quietly moved on without sharing any feedback.

This isn’t a story about backwards-thinking people who aren’t ready for change. These are excellent engineers at a company that’s been at the center of technology for decades – if you saw this name on a resume, you would immediately consider interviewing the person. And yet, for most of them, AI coding tools remain something they’ve heard about but haven’t really integrated into how they work.

If that’s true there, imagine how much of a challenge this is everywhere else.

The bubble is smaller than you think

If you’re reading this blog, you’re almost certainly deep in the AI bubble. You’ve probably used Claude Code or Cursor to build something from scratch on a weekend and told everyone you know about it. You’ve definitely seen the LinkedIn posts — the ones breathlessly explaining that a repo with Claude Code skills for go-to-market is going to revolutionize the way you do sales. We get it. We write this blog, after all.

The problem is that spending time inside the bubble makes it easy to mistake the bubble for the world. The users who are most excited about AI — the ones tolerating rough edges, experimenting constantly, building their own workflows — are a small fraction of the people who are eventually going to need to use your product. And crucially, their enthusiasm can paper over a lot of gaps. If someone is genuinely excited to use your product, they’ll push through a confusing onboarding flow, forgive a clunky UX, and figure out the right prompting pattern through trial and error. You can build a real business on these true believers for the first little while – and they’re genuinely wonderful people to work with.

But you will eventually have to account for the rest. And we think most AI companies aren’t building for them.

Why the gap is a product problem, not a people problem

The instinct when faced with reluctant users is to assume they’ll come around — that adoption is just a matter of time, and the product doesn’t need to change if its working well for the true believers. AI adoption has a different quality from previous technology changes, and the reason is rooted in something more fundamental than UX.

User identity

AI tools attack user identity in a way that most software doesn’t. Think about what it means to be a software engineer who has spent years taking pride in the quality of the code they write. Now an agent is generating code at 10x the speed, and most of it looks like slop to them. For many senior engineers, it’s probably true – they could write better code if they took their time. More importantly, that code is an affront to how they see themselves professionally. They’re never going to merge this code without a close inspection, no matter how many benchmark improvement graphs you put in front of them.

But here’s the flip side, and it’s where the real product opportunity lies: The best AI products have the opportunity to actively reinforce user identity too. Done right, your product doesn’t make the careful engineer feel replaced; it makes them the most forward-thinking engineer on their team. It doesn’t make the skeptical PM feel like they’re cutting corners; it makes them the person who shipped a working prototype before anyone else had finished writing the spec. The goal isn’t to neutralize the identity question. It’s to make your user the superstar of their organization, in a way that feels consistent with who they already are.

Getting there requires meeting users where they are and bringing them along gradually. The coding agent world offers the clearest model of how this works in practice. The trust that made widespread agent adoption possible wasn’t built by agents — it was built by tab completion. Engineers who would never have handed off a function to an autonomous agent were perfectly comfortable accepting a 1-5 line suggestion. That comfort, built up over thousands of small interactions, is what created the foundation for people to trust agents with more. It’s not a coincidence that the products with the deepest agent adoption today are the ones that also had the best tab completion two years ago.

Editorial judgment

The corollary to meeting users where they are is not overwhelming them with everything at once. This has always been true in product, but AI has made it a more acute problem. The ability to ship new features quickly — which AI has genuinely accelerated, and AI products are taking full advantage of — makes it easier than ever to pile on functionality before users have developed trust in the core experience.

A reluctant adopter who opens your product and finds a sprawling set of capabilities they don’t understand isn’t going to dig in and figure it out. They’re going to confirm their suspicion that this thing isn’t for them, and they’re not coming back for a long while. The same logic applies at a feature level: if a user is still getting comfortable with tab completion, immediately introducing an autonomous agent mode isn’t going to accelerate their adoption — it’s going to spook them. Doing less, more reliably, for the right person at the right time is how you earn the trust that eventually lets you do more.

This concern applies in many other domains. We see every day that most engineering leaders are only now coming around to the idea of having an AI SRE help them manage production software – and while these folks are figuring out what they want, vendors are promising that they’ll autonomously make changes to your production infrastructure while you’re still asleep. That might sound exciting to the most ardent believer, but it sounds like absolute insanity to most people. Most products in the space haven’t figured out the balance yet.

The maturation moment

There’s a useful historical parallel here. In the early days of cloud computing, the customers were startups and tech-forward companies who were willing to figure things out as they went. AWS in 2008 was powerful but rough, and the people using it had enough technical sophistication and tolerance for ambiguity to make it work. Then came the maturation phase: large enterprises, regulated industries, banks, healthcare systems. These customers had different expectations, different risk tolerances, and different definitions of “good enough.” The cloud providers that thrived were the ones that understood the product had to change — not just get cheaper or faster, but fundamentally adapt to a different kind of user.

AI is entering the same phase. The early adopters — the ones giving talks to thousands of engineers, the ones building apps on weekends and posting about it — have gotten the industry here. The next phase of growth is the middle 50%: the users who are curious but skeptical, willing to try but quick to bounce, not interested in figuring out the right prompting strategy on their own. That is the audience that’s going to determine whether AI actually delivers on the productivity promises that have been made on its behalf.

What makes this moment unusual is that unlike the cloud, AI seems to be compressing the early adoption and maturation phases together. The gap between this is a tool for enthusiasts and this needs to work for everyone is closing faster than anyone expected. For product builders, that’s both an opportunity and a warning. The companies that figure out how to cross the chasm early are going to have a significant head start on the ones that are still optimizing for the bubble when the rest of the world shows up.