You can’t build a moat with AI (redux)

Using LLMs doesn't make a product special anymore

Last spring, we wrote an article called You can’t build a moat with AI. That post argued that prompt engineering, while important, would be difficult to defend over time given how easy it is to experiment with LLMs. As a result, you have to focus on the quality of data your application has access to and your use of that data to differentiate yourself.

Everything we said in that post has held true, but the release of models like DeepSeek R1 and o3-mini have brought the concerns about AI applications’ moats back to the forefront. In the last few weeks, we’ve heard many forms of the question, “If LLMs keep getting cheaper and better, won’t everyone’s application get better? If so, how do you differentiate?” It’s worth it for every application builder to think through this in the more detail.

In short, unless you’re building or hosting foundation models, saying that AI is a differentiating factor is sort of like saying that your choice of database is what differentiated your SaaS product: No one cares.

Source: DALL-E 3.

As a baseline assumption, better models are a rising tide that lifts all boats. It’s generally a good thing for AI applications when more powerful models come out because it gives you a more powerful engine to build on, but this isn’t the basis on which you’re going to build a moat. In the immediate frenzy after a new model is released, one team or another might figure out a better way to use that model, but that advantage is going to evaporate extremely quickly. There’s far too much useful content out on the internet for those advantages to last. This is basically the same as the prompt engineering moat we discussed above. In other words, if you’re relying on that as your differentiation, you’re going to get quickly steamrolled by the competition.

Of course, the existence of a good tool isn’t sufficient — how you use the tool matters just as much. Again, you can’t credibly claim that your application is “smarter” than the competition’s, so you’re going to have to look elsewhere for your differentiation. We’ll refer you back to last year’s post for a walkthrough of how you can build a technical moat, but in light of recent model advancements, we’re thinking about this from a product and market perspective.

We think defensibility comes down to three things.

User experience. Sprinkling AI on your application for the sake having AI is something that we’ve railed against many times, so we’ll spare you the sanctimony. As we’ve talked about a couple times recently, we believe that most user experiences should be rethought for AI applications. There are many scenarios in which we’ve had to rely on user input previously that we can now infer. In many ways, the whole point of AI applications is that it should feel like magic because something that you previously had to do by hand is now fully automated with believable intelligence. If you’re thinking about taking traditional forms of UX and adding AI to them, that’s an okay starting point but not a defensible moat. On the other hand, if you’re thinking about how you can meet the user where they are, integrate into their workflow, and get tedious busy work out of their hair seamlessly, you’ll find a way to change the UX. Good UX, of course can be copied, but the teams that are capable of innovating once will do it again (see: Snap). In the context of enterprise products, the focus should be on delivering useful work.

Integration and workflows. We’re in the business of selling enterprise products, and many of you probably are too. We like to joke that every enterprise company either becomes a database company or an integrations company, and it’s clear that most AI applications companies are going to be integrations-heavy. In the previous paragraph, we touched on the importance of delivering work and snapping into the user’s works — that means that you’ll need to integrate with the customer’s messaging tool, document system, task tracker, and so on. This might sound tedious (and it can be!), but it’s also critical. Products that are able to integrate deeply into a customer’s workflow are incredibly sticky. You’re providing them so much value that they want you to be a part of everything they do — if they’re ever considering replacing you with something else, that product will have to be at least as valuable and justify the cost of re-integrating with everything you work with.

Data in, data out. Everything in AI always comes back to data. We talked about the use of data extensively in our previous post. To summarize briefly, simply having data is not enough. The hard part is using the right data at the right time (hint: vector search is not enough). From a business perspective, data out is just as important as data in. At RunLLM, we’re now answering over 100K questions per-month. We can use that data to help our customers in many ways: understanding what features their customers are using, tracking the popularity of different workloads, identifying bugs, and so on. With some labeling, we can also use that content to help ourselves understand what kinds of questions we could do a better job of answering. Not every application is going to do all of those things on day one, but gathering the data over time builds an asset that you will benefit from tremendously in the long run. How you gather that data and how you use it over time will make a huge difference.

---

Ultimately, our opinion is that AI is no longer an interesting thing to talk about in terms of differentiation. At best, it’s a diminishing advantage, and its worst, it means you’re not planning for the future. Every application should probably be using AI in some form or the other, and the question is how effectively you understand your user’s needs and how effectively you’re able to shape your application to meet those needs. AI has so quickly become mainstream that it should now be considered a standard tool — a carpenter doesn’t brag about their hammer, and you shouldn’t brag about your use of GPT.