The rise of the AI marketplace

Businesses such as OpenRouter are thriving as companies try to cut token costs

A UNIT OF account is a useful thing. Even before money was common, people tried to come up with standardised ways of accounting for things. In “The Iliad” the value of armour fashioned from bronze or gold was expressed as a number of oxen (nine for bronze, 100 for gold). Fur pelts, giant stones and piles of salt have all been used as measuring standards.

The universe of artificial intelligence is governed by its own unit: the token. (Unsplash)

The universe of artificial intelligence is governed by its own unit: the token. These are snippets of text, typically four characters, that an AI model reads or responds with. The trouble with tokens, however, is that they do not work quite as neatly as cows or pelts did. For one thing, it is unclear in advance just how many a given task might consume. Model-makers also charge dramatically different prices: a popular model offered by DeepSeek, a Chinese startup, is about one-fourtieth the price per token of a cutting-edge one produced by Anthropic, a top American lab.

For firms using ai, this is a recipe for frustration. A simple task, such as asking a model to process invoices, could rack up a huge bill. To insure against this, some businesses cap how many tokens employees can consume. But that may hinder productivity. This muddle has birthed a new kind of business: the model marketplace.

The best known of these marketplaces is OpenRouter, co-founded in 2023 by Alex Atallah. After ChatGPT was released, Mr Atallah began to wonder what kind of world lay ahead. “My biggest question at the time then was: could this be a winner-takes-all market where OpenAI dominates,” he says. He doubted such an outcome, leading to the conclusion that marketplaces would play a central role in the AI industry.

OpenRouter lets customers easily switch between models. If Anthropic’s Claude is down, they might use one from OpenAI. Or they might swap when a lab is offering a bargain on tokens. OpenRouter charges a small fee for use routed through its platform. Business has been so good that in August Stripe, a payments giant, agreed to buy the marketplace. It will reportedly pay $7.5bn for a firm that is barely three years old.

OpenRouter is not the only model marketplace. Similar services are operated by the big cloud-computing providers. Amazon offers a range of models via Bedrock, billed through existing cloud-service agreements; Google offers the same through its Vertex AI service. But many customers view such offerings with suspicion: the cloud giants, which build their own models and invest in the frontier labs, are hardly neutral.

OpenRouter first gained a following among developers as it offered a way to play around with lots of different models and a means for quickly gauging which would perform best. After the release of ChatGPT, many AI labs took to “benchmaxxing”—making models that could pass certain tests, such as the sat, with flying colours, but struggled with more useful tasks. On OpenRouter’s marketplace customers vote with their dollars, creating a fairer ranking system.

More recently companies have turned to OpenRouter as a tool for controlling costs. Tellingly, it is the cheap systems offered by DeepSeek and two other Chinese model-makers, Z.ai and Xiaomi, that dominate token consumption on the platform (though an OpenAI model is also currently doing well).

Even so, Mr Atallah argues that many businesses remain unsophisticated in their approach to token use. He recommends to his enterprise clients that they divide up tasks into low, medium and high “entropy”, based on how open-ended the work is. Cheap models are often sufficient for low-entropy tasks, such as deciding which model should review some code, and plenty of medium-entropy tasks as well, such as actually reviewing that code. Frontier models can be reserved for high-entropy tasks, like writing the code from scratch.

“The mistake I see a lot of companies make is they just don’t decompose all of these things. It is all the same to them,” says Mr Atallah. “That’s just a massive waste of money.”

To track the trends shaping commerce, industry and technology, sign up to “The Bottom Line”, our weekly subscriber-only newsletter on global business.