Why the Cheapest LLM Doesn't Win: What 100 Trillion Tokens Reveal About Model Routing
In October, AIEEV joined FIX 2026, the largest technology fair in Daegu, held at EXCO. This year's theme was "AI, Beyond Technology, Into Everyday Life." With 588 companies taking part, it was the biggest edition yet. We returned for a second year and ran a booth in the PNP Korea Startup Arena.
AI trends move fast. Even so, the difference between last year and this year was striking on the show floor. Last year, most AI services we saw were assistants that supported people at work. This year, more of them had moved into decision-making. They ran systems, operated equipment and controlled processes on their own. As AI takes on more responsibility, the infrastructure, data and security underneath it matter more too. In this post, we look at what that shift asks of the infrastructure layer and share research that can help teams building AI services.
🧭 From Judgment to Simple Processing: Matching Models to Tasks
The decision-support services we saw at the event don't need the same level of reasoning for every task inside them. Sorting and summarizing the material behind a decision is a different job from weighing that material and comparing options. Take an AI that helps with customer support. When it classifies inquiry types, processing cost matters most. When it suggests how to handle a complex complaint, accuracy and context matter most. When it talks with customers in real time, it also needs fast responses. Even within a single service, the priority between accuracy, cost and speed changes from task to task. Model routing sends each request to the model that fits those differences. The diagram below is a simplified example of how a model is chosen based on the request and its operating conditions.
Routing sits at the front of this flow. It looks at how complex a request is, how much can be spent on it and how fast it needs an answer. Then it decides whether to send the request to a high-accuracy model, a cost-efficient model or a low-latency model. Tools like LiteLLM, OpenRouter and Not Diamond handle this role.
Enterprise numbers point the same way. In a 2025 a16z survey of 100 enterprise CIOs, the share of companies using five or more models rose from 29% to 37% in one year. Running multiple models in production has become the norm. The main reason is performance differences between tasks, more than avoiding vendor lock-in.
📊 The Numbers: Heavier Requests, Faster-Moving Prices
The data makes it clearer why routing has become a hot topic. Let's look at a study of real usage on OpenRouter, a leading routing platform with more than 300 models.
First, each request is getting heavier. The share of tokens processed by reasoning models went from negligible in early 2025 to over 50% within a year. Average tokens per request roughly tripled, from under 2,000 in late 2023 to over 5,400 in late 2025. Coding requests ran 3 to 4 times longer than general ones. The researchers read this as a sign that agentic inference, with multiple steps and tool calls, is becoming the default.
Second, there are more options. Open-source models handled about one third of all tokens by late 2025. Chinese open-source models started at a 1.2% weekly share in late 2024. They climbed close to 30% in some weeks and averaged about 13% over the year. The setup where one or two models handled everything is giving way to one where teams pick from many.
Third, prices are moving very fast. According to Epoch AI, the inference price for the same level of performance fell between 9x and 900x per year, depending on the benchmark. A model mix that was optimal a few months ago may not be optimal now. The best choice shifts with every request and over time, and it is hard for people to keep up.
Researchers have studied this problem for years. Stanford's FrugalGPT study noted that LLM API prices can differ by up to 100x. It showed that calling multiple models in sequence could match GPT-4 performance at up to 98% lower cost. LMSYS's RouteLLM study had a router switch between a strong model and a weak one. On some benchmarks it cut costs by more than 2x with no loss in quality. That research has now reached the market as real tools like LiteLLM, OpenRouter and Not Diamond.
👀 What We Noticed: Who Picks the Model Is Changing
From a technical view, routing is a tool for efficiency. From a market view, it means something more. The choice of models and infrastructure is moving from people to software.
In the past, people compared vendors, picked one, signed a contract and stuck with it for a while. Brand awareness, sales relationships and the effort of switching all played a big role. With routing, the choice happens on every request, and the criteria are conditions set in advance. People set the conditions. The router does the choosing.
Electricity markets already work this way. They don't care which company owns a power plant. They dispatch the generators that can meet demand at that moment, starting with the lowest cost. This is called the merit order. A router does the same thing. For each request, it picks the most efficient option that meets the conditions.
⚖️ But Routers Don't Pick the Cheapest Option
So does the cheapest provider simply win? OpenRouter's data says no. When the study compared price and usage across models, the correlation was almost flat. A 10% price cut raised usage by only about 0.5% to 0.7%. Price alone does not move demand much.
Instead, the market splits into clear segments. The researchers grouped models into four types. Premium workhorses cost about $2 per million tokens and see heavy use. Efficient high-volume models cost under $0.40 and also see heavy use. Long-tail models cost a few cents but see little use. Premium specialists cost about $35 and serve a small number of high-stakes tasks.
This split mirrors a pattern where closed models handle high-value work and open-source models handle cheap, high-volume work. The researchers concluded that LLMs do not yet behave like a commodity. That is what a router does in practice. It doesn't send every request to the cheapest model. It sorts requests into a few lanes and sends each lane where its conditions are met. Competition happens inside each lane, not across the whole market.
🧩 Each Lane Has Its Own Winning Conditions
In lanes where results carry high value, quality and trust set the bar. The researchers explain why expensive models keep high usage. If the code saves a developer an hour, a few dollars in API costs don't matter.
In high-volume lanes, price matters. One enterprise leader in the a16z survey noted that for most tasks all models now perform well enough, so price has become a much bigger factor. For simple or internal tasks, model choice effectively comes down to cost.
Still, the research shows that cheap alone is not enough. Many long-tail models saw little use even at near-zero prices. They lacked the performance for the task or their output wasn't stable, so developers hesitated to integrate them deeply. That is why the researchers see the yardstick shifting from cost per token to cost per successful result.
A simple calculation shows the difference. Option A costs KRW 100 per call but succeeds 60% of the time. It takes 1.67 calls on average to get one success, so each success costs about KRW 167. Option B costs KRW 140 per call but succeeds 95% of the time. Each success costs about KRW 147, which makes it the cheaper choice. In a real service, retry latency and human review time add up, so the gap grows even wider.
🔒 The Opposite Force: Once It Fits, Teams Rarely Switch
Routers choose per request, but not every choice is made fresh each time. The same data also shows a force pulling the other way. The State of AI researchers call it the "Cinderella glass slipper effect." Early users who found that a new model fit a specific task right after launch kept using it much longer than users who arrived later. Once a task and a model click, workflows and habits build up on top of them. The a16z survey also found that as agent workflows get more complex, switching models takes significant engineering time.
So per-request choices are moving to routers. But which options make the router's shortlist is still decided when a team designs its workflow. Becoming a proven option at that moment is what leads to being chosen again and again.
✅ Three Things to Check Before You Add Model Routing
As we saw above, the models and infrastructure on your shortlist are set when you design your workflow, and they rarely change after that. So before choosing a routing tool, start by looking at the requests in your own service. If you are building AI services on LLM APIs, check these three things first.
Have you split your requests into lanes? Start by finding which tasks get called most often and which model each one uses today. Costs are usually driven by the most frequent task, not the most expensive one.
How long does it take to switch models? If you can't swap a model with one line of config, it is time to consider a routing layer or a standard API. In a market where prices move by tens of times a year, switching cost is opportunity cost.
What do you measure cost against? Comparing price per token leaves out the cost of retries and reviews. Recalculate using cost per successful result, and other options may come into view.
🌏 What AIEEV Is Focused On
AIEEV's Air API and Air Cloud sit closest to the high-volume lane on this map. We connect distributed GPUs without owning physical data centers, which gives us a head start on cost. But as the data shows, being cheap is not enough to get chosen, even in this lane.
In September, we tried splitting lanes ourselves. While building an internal research agent, we assigned a different model to each stage. It made 52,406 calls and cost $2.41 on Air API. This wasn't dynamic routing that decides per request. It was the simplest kind of split, with a model fixed in advance for each stage.
The results pointed the same way as this post. The classification stage made up 83% of all calls, so we gave it a small 9B model. That stage still accounted for half the total cost. The extraction and summarization stages needed more language understanding, so we used a 35B model there. Running every stage on the 9B model would have cost $1.59. We chose to spend $0.82 more for quality. When you split lanes, the most frequently called lane sets your cost. That frees up the other lanes to choose for quality.
Air API offers an OpenAI-compatible API, so you can connect by changing only the API key and base URL in your existing code. According to the State of AI study, Asia's share of AI inference spending more than doubled from 13% to 31% over the study period. Serving that demand reliably from nearby takes more than low prices. It also takes steady performance on metrics like latency and availability.
That is the direction AIEEV is working in. We are refining serving quality so the same request gets consistent results. We are also researching routing technology that matches each request with the right model and compute more efficiently. Our next goal is to be the infrastructure that lowers your cost per successful result.
References
a16z, How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025 (2025.06)
Epoch AI, LLM inference prices have fallen rapidly but unequally across tasks (2025.03)
Ong et al. (LMSYS), RouteLLM: Learning to Route LLMs with Preference Data (2024)
AIEEV Blog, We Processed 52,406 API Calls for $2.41. Here's the Stack. (2026.09)
