A theory of the AI market

Dangerous (but fun!) predictions about AI adoption

It’s always tempting to predict what the world is going to look like in 1, 2, or 5 years — especially given how fast things are changing with AI. We’re aware that predictions are difficult, so we tried to avoid making broad claims and instead focus on the narrower, tactical things that we’ve learned from our experience. But today, we’ve decided to go out of our comfort zone to try to pull together multiple lines of thought that we’ve been exploring for a few weeks now. As we’ve spent more time with customers and seen how the market has evolved, we’ve begun to formulate a theory of how the AI market will solidify in the next couple years.

Our theory of the AI market has two main aspects. First, customers are going to increasingly want to see value from AI over the next 12-24 months, which is going to benefit adoption of AI applications. Second (and as a consequence of the previous point), we believe that value is going to accrue to the ends of the stack — core model providers on one end and applications on the other.

Source: Ideogram. We had to switch away from DALL-E 3 to get better text rendering.

Before we dive in, it’s worth saying that we usually don’t like grand unifying theories of things in any context. Occasionally (e.g., Sapiens by Yuval Noah Harari) you can put together a nice narrative that accurately reflects the state of the world. Oftentimes (e.g., the subsequent books Yuval Noah Harari has written), however, you get theories that are frustratingly incomplete. They either ignore obvious counterexamples to maintain a tidy narrative or they include so many edge cases and nuances that they become unhelpful — that is, an accurate map of the world has to be the size of the world.

Nevertheless, we think it’s worth sharing our thinking for two reasons. First, we’re placing our own bets with RunLLM based on this thinking, so we’re putting our money (time) where mouths (blog posts) are. Second, even if we’re wrong — which we very well might be! — it’s worth tracking the evolution of our thinking over time, so we can be less wrong the next time around (and it’s fun to bring you all along).

With all those disclaimers out of the way, let’s dive into our theory of the AI market.

Part 1: How enterprises derive value from AI

It’s pretty safe to say that there’s a massive AI bubble right now. The bubble is going to burst sooner or later — both in the public and the private markets. When that happens, the worst case scenario is that AI investment dramatically plummets. We generally think this is unlikely to happen because there’s simply too much investment and too much promise for customers to give up completely.

What we think is more likely is that enterprises are going to focus on how best to derive value from AI products — either by increasing top-line (revenue) or bottom-line (cost-cutting). This focus would have emerged naturally anyway, but the state of the hype cycle is going to accelerate the transition. Realistically, we believe the short-term wins are going to be in cost-cutting, because AI products will help automate tedious work that people currently drag their feet to do. Most importantly, these products are easier for enterprises to evaluate because time-saved is an extremely easily measured metric (even if cost-cutting is not where products want to be long-term).

There will be two approaches to finding value from AI. Companies with the technical know-how and organizational capabilities will hire or allocate staff to build AI applications — whether it’s automating support, writing code, or doing sales outreach. These homegrown solutions will be tailored to the needs of these few organizations and extremely high-value, but there aren’t very many large companies today that have the expertise to ship reliable AI applications. For this approach to be economical, you’re likely going to need to build and maintain many such applications with the same team.

Everyone else will turn to vendors. The time-to-value and likely even the total cost of ownership for a purchased solution over time will be significantly lower if you’re not an AI-first company. The companies and markets that succeed will be the ones that are the hardest to hire in and are the ones that are comparatively amenable to automation. As is now perhaps common knowledge, tier 1 support (e.g., order refunds and password resets), sales prospecting, and code generation will likely be the most sought-after markets. The narrative around the products that succeed will generally be one of process automation and consistency — likely accruing to the bottom line in the short-term.

This is a prediction about the short-to-medium term — the next 2 or so years — rather than a statement about the AI market over the next 5 or 10 years. But the opportunity for application-level vendors over the next 2 years is quite big — ****just look at the Series B that Decagon (one of the leaders in tier 1 support automation) raised. Its very likely that many markets will see a land grab, as the wisdom today seems to be products are relatively sticky (since comparing quality is hard).

The hard part of the sell is one we haven’t yet talked about: proving that your AI application is actually going to deliver on its promise. In the absence of twenty years of SaaS usage data and metrics, many AI applications are relying on the promise of automation and scale today, but the hypothetical bubble burst we’re talking about is going to force enterprises to measure with actual value. That means mid-level infrastructure might get cut (more on that below), and application vendors are going to have to find the right metrics to convince prospects that their value is real. We don’t know what those metrics are yet, but it’s something we’re spending a lot of time on; we think it’s one of the next key unlocks to scaling an AI product.

As a note for a future post, we’re also very interested in the transition from bottom-line products to top-line products as well, but we don’t have anything well-formed to say on that (yet).

As a quick summary, we believe:

1. Enterprises will focus on applications to find value (mostly cost savings) over the next 1-2 years, and while a few will try DIY solutions, most will turn to vendors.

2. The companies that succeed will provide automation for tedious work.

3. Finding ways to measure value is incredibly important (and incredibly hard).

Part 2: A barbell distribution of value

If things play out the way describe above, there will be huge implications for which submarkets are the most valuable. Regardless of what happens, the foundation model providers will be minting money. Whether you’re building in-house solutions are buying from a vendor, a large portion of the money will be flowing back to OpenAI, Anthropic, and Google. This is probably an uncontroversial statement, so we won’t belabor the point.

If our guesses in part 1 are correct, there will likely also be a ton of enterprise value created in the application layer. However, unlike the foundation model space — where value will accrue to very few winners — there will likely be many winners in the application layer. In the same way that technical support engineers do a very different job from someone doing B2C support, there will be applications that specialize in each one of these in the short term that should likely be successful. A Salesforce-sized behemoth might emerge over the next decade, but the viability of that model depends both on the evolution of the technology and governmental regulation. We’ll put that aside for now.

The products that are less immediately obvious to us are those focusing on mid-level infrastructure. Data systems, monitoring infrastructure, and AI development frameworks are potentially useful for anyone building in-house solutions, but again, there are relatively few companies who are equipped to do that today. Meanwhile, most startups we know — RunLLM included — have homegrown simple versions of many of these components over the last 12 months on top of existing cloud primitives. The main concern we have is that the value of mid-level infrastructure will not be immediately obvious to most enterprises, which puts those products in a precarious place in our hypothetical world.

One of the main reasons why we’re comparatively bearish on mid-level infrastructure is that it feels too early to really know what kinds of tools will generalize well. Given how fast foundation models are changing and best practices are emerging, it’s difficult to imagine a platform that’s going to generalize across many use cases — which is why most startups have found it easier to roll their own lightweight stack. Of course, this will change over time, but it feels today like we might be trying to build Heroku before figuring out what web applications will look like. That said, we’re close friends with many of the these teams, and we’re confident they will figure it out over time — but the broader market has many open questions.

The biggest exception to this rule is model inference: GPU prices are still high enough and utilization is still low enough that it doesn’t make sense to do inference ourselves. However, we’ve found that most of the providers in this space are competing on price. This is great for us as consumers, but it creates a race-to-the-bottom dynamic which can be dangerous for businesses. There will be winners in the space, but it’s hard to predict who they will be right now.

We believe this will create a barbell distribution of value. At the bottom of the stack — the foundation model and infrastructure layer — you’ll find the largest players and the most value created. At the top of the stack, applications will aggregate a ton of cost savings and eventually revenue generation. The middle of the stack will be comparatively thin.

It’s worth noting that this is how the cloud + SaaS played out as well. The public cloud providers and lower-level infrastructure generated a ton of value, and SaaS applications of course were very successful (and highly varied) as well. The middle of the stack — monitoring tools, data analytics companies, etc. — were relatively thin earlier on in the lifecycle and only began to broaden as the market mature in the mid-2010s. In other words, Salesforce, Zendesk, and Hubspot all generated a ton of value well before Datadog did.

The same will likely be true here, and the million dollar question will be when the market is mature enough to support large-scale mid-level infrastructure.

As a quick summary, we believe:

1. Foundation model companies will be successful regardless.

2. The application layer will likely have many winners which will aggregate a ton of value as well.

3. Mid-level infrastructure is not as compelling today because of how immature the market and technology are; this will change over time, but the speed of maturation is a large open question.

Conclusion

You probably disagree with some of what we’ve said here. That’s always the risk when you’re trying to draw broad conclusions, so we’d love to hear your thoughts on anything that we’ve discussed. We generally believe in strong opinions, loosely held — this is what we believe about the world given our experience thus far, but we’ll change our minds if we’re presented with new evidence or a better argument.

That evidence might not emerge immediately — AI is still incredibly early — so we’ll try to find the time in 6 or 12 months to revisit this and see how well our intuitions played out. We’ll also do our best to explain why we missed whatever we (inevitably) got wrong. In the meantime, tell us why we’re wrong!