The hidden costs of AI

If you’re marveling at how cheap it seems to use AI, or how accessible it’s become to ask complex questions and get coherent answers, hold onto that thought—because the true costs of AI are anything but cheap. What we’re witnessing with hyperscale AI companies like OpenAI, Google, and Microsoft is a play straight out of Silicon Valley’s disrupt-at-any-cost handbook. The same approach used by Uber, Amazon, and others—where users get subsidized services until they’re too hooked to resist price hikes—is being deployed here, with AI’s hidden costs being masked or deferred. We had Big Tech, this is Big Tech 2.0: new tech, same playbook.

In this game, there are three ticking bombs of AI costs industrials need to investigate and derisk as soon as possible:

1. The subsidization trap

Right now, AI services like ChatGPT feel affordable, even free at the basic level, but this setup is a long game for Big Tech. The actual costs behind these platforms are astronomical—estimates for just one ChatGPT query can range from a few euros to possibly hundreds. High-quality training alone, which involves vast amounts of computing power, expensive hardware, and highly skilled engineering talent, costs millions. But just like Uber’s initial rides, which cost much less than the true market rate, AI services are being subsidized so that we, the users, become reliant on them.

Imagine if ChatGPT queries were priced at their real cost from the start. What if it was €2 per request, or €20... or €200? Would casual users and even most companies still engage? Probably not. By disguising the actual cost, hyperscalers can edge out smaller players, making it impossible for anyone else to offer competitive services at this scale. Then, just when the competition is cornered, they will incrementally raise prices or even change usage policies, reclaiming the costs they’ve deferred. And we’ll pay because, at that point, we’ll have no other viable options.

2. The accuracy plateau problem

Even if you accept the subsidized AI business model, there’s a more sobering problem in the technology itself: its potential accuracy plateau. Current AI models, while impressive, don’t guarantee perfect accuracy—and we’re seeing diminishing returns in improving these systems further. Today, the best large language models might get answers right 80% of the time in general-purpose applications. But what if we’re stuck here, unable to reach the elusive 100%? A self-driving car that turns right 99.9% of the time when it needs to turn right is practically unusable on real roads; the same applies to AI which suggests medical treatments or automates financial decisions. Reliability thresholds can be game-breaking.

Industries are beginning to assume that AI accuracy will improve over time, akin to Moore’s Law in computing. But AI may defy this principle. If getting from 80% to 85% accuracy takes another decade, and each incremental improvement becomes exponentially harder and costlier, companies could be investing in a future that doesn’t materialize as they envision. Hallucinations—the tendency of models to make things up or interpret ambiguities incorrectly—are not a trivial bug; they’re fundamental limitations that might prove impossible to resolve fully. If we can’t overcome this, and if models remain “black boxes” whose outputs we don’t fully understand or control, then the hidden cost isn’t just monetary; it’s operational risk and trust.

3. The arms race for proprietary AI models

Assuming we navigate subsidization and plateauing reliability, there’s a third elephant in the room: the cost of going proprietary. Right now, companies that want to avoid sharing the same models as their competitors are facing a tough choice. Either they pay hyperscale companies for custom models, essentially locking themselves into a vertical ecosystem, or they attempt to build their own, a prohibitively expensive and technically challenging task. To put this in perspective, major investment banks spend billions annually on trading infrastructure to maintain a marginal edge in high-frequency trading. For AI, the scale of investment already surpasses even that.

Consider what it takes to build a proprietary large language model. Companies would need dedicated data centers, specialized chips, and a continuous stream of data inputs to train and retrain the models. And then there’s the personnel: scarce AI talent doesn’t come cheap, and retaining experts who understand the complexities of such advanced systems is another uphill cost. Few companies outside the hyperscaler firms can realistically invest this way, meaning most will remain dependent on generic models, conceding strategic control to the very players who are hiding the true costs. A vicious (but perfect) lock-in.

AI will not be a revolution for everyone

These hidden costs of AI—the deferred true price of subsidization, the operational risks of a plateau, and the near impossibility of going proprietary—reveal an unstable market structure. While Big Tech 2.0 continues to offer seemingly affordable AI and a promise of ever-growing accuracy, the long-term expense and limitations are quietly accruing in the background. If we’re going to rely on these systems, we must first understand and plan for the costs we don’t yet see but that will inevitably emerge. Companies will have to realize that dependency on the current hyperscalers for AI could be far more expensive—and strategically constricting—than anticipated.

Not everyone will be able to play this game...

EDIT - Oct 31, 2024

Right on time, Bloomberg published a long and detailed article about why Amazon is struggling to replace Alexa with a proper LLM AI model. Among other things:

Shifting to pretrained AI models meant Alexa could handle infinitely more complex questions on its own but risked becoming less reliable for basic tasks, such as setting kitchen timers or fishing a one-off answer from a plugged-in database. For example, when Jassy tested the Alexa AI prototype that summer of 2023, it wasn’t able to deliver accurate football scores on the spot because it was tapped into a generic language model, not real-time sports info. - Alexa’s New AI Brain Is Stuck in the Lab

This is the recap of a discussion I had yesterday with a London-based S&P analyst on these topics. I was not planning to write about it, but getting asked "What do you think about GenAI development bottlenecks?" helped materialize the key worries I was starting to have about what OpenAI & Co are doing. Thanks Iuri.