Why Compute Is Scattering Again
If you have been following AI over the past week, you have probably come across at least one of the following stories.
Anthropic, which is preparing for an IPO, is reportedly weighing whether to present investors with a potential market of more than $30 trillion
The "software apocalypse" argument that AI will dismantle the existing software industry, and the counterarguments to it
NVIDIA unveiling an entry level edge AI module that delivers 78 TOPS of compute
Capital markets, software, cloud infrastructure, edge hardware. At first glance these come from different fields. Placed side by side, though, they all converge on a single question. Where and in what structure will the growing volume of AI computation be processed? In the end it is a question about cloud strategy. For the past decade computing moved in one direction only. Scattered servers gathered into data centers, and data centers gathered again into a handful of hyperscale regions. Economies of scale clearly worked, so it was a natural progression. Yet the signals of 2026 point the other way. Why is compute scattering again, and why now?
📈 Signal 1. The market's unit of measure moved from software to labor
Anthropic is reportedly considering presenting a total addressable market of more than $30 trillion in its IPO prospectus. What deserves attention here is the method of calculation. Rather than using a specific industry or the software market as the baseline, it counts the entire body of work that AI models can perform as the market. Human labor itself, not software licenses, is treated as the potential market. This figure is of course hard to accept as a realistic revenue forecast. Anthropic's revenue in the second quarter of 2026 was $11.6 billion, and according to FactSet the combined revenue of the 191 technology companies in the S&P 1500 came to $2.4 trillion last year. $30 trillion goes far beyond the size of the entire technology industry today. So the point worth noting is not the accuracy of the number but the fact that the basis for sizing demand is changing. The potential market is no longer the software market but the full range of human work that AI can take on.
💸 Signal 2. What the apocalypse actually pointed to was a shift in budget lines
As AI began to treat human work itself, rather than the existing software market, as its potential market, the debate over the future of the software industry intensified. Earlier this year the "software apocalypse" view spread: AI agents would replace the functions of existing SaaS, corporate IT budgets would move from applications to AI infrastructure, and the traditional software industry would contract. Counterarguments followed. Dan Ives of Wedbush distanced himself from that outlook. On CNBC he said demand for hyperscalers, AI cloud providers and NVIDIA remains strong, and pointed to the present moment, as enterprises begin converting AI investment into real commercial use cases, as an important inflection point.
More interesting than the debate itself is the premise both sides share. Those arguing for the software apocalypse and those rejecting it agree that enterprise IT spending is moving from the application layer to the infrastructure layer. The only difference is their judgment on whether that shift will bring down existing software companies. The shift leaves a very concrete result for teams building AI products. When revenue arrives as a per user subscription while cost goes out as a per execution inference charge, the unit price of infrastructure becomes the product's margin. Infrastructure cost used to be a line item to manage. Now it is a variable in the business model. The same change shows up inside organizations. As covered in an earlier post, the share of companies with a FinOps team managing AI cost rose from 31% in 2024 to 98% in 2026, and 78% of those teams sit under a CTO or CIO rather than under finance. Cutting cost has become an engineering decision rather than an accounting task. Oracle made the same point. Karan Batta, senior vice president of Oracle Cloud Infrastructure, said inference "will be the largest operating cost for many customers." He added that enterprises are asking for optimization across token cost, GPU utilization, network efficiency and operational simplicity.
🌐 Signal 3. New demand for distributed cloud
Batta's point does not end with cost. Lowering the unit cost of inference starts with deciding where inference runs, and his starting point for that question is the fact that enterprise data does not sit in one place. "Most enterprise data is not in a single cloud. It is on premises, in Oracle databases, with other cloud providers and in neoclouds." Customers, he explains, "want to keep their data where it already is." The direction is to send AI to where the data lives rather than move data to where the AI lives.
The reasons are regulation and data sovereignty. In regulated industries the physical location of data is not negotiable, and no company can build a separate AI strategy for every location. That creates a requirement: locations may be scattered, but governance has to stay consistent. The nature of the problem has changed as well. Batta put it this way: "It has moved from how to build an AI model to how to operate AI across the entire organization." The stage where model choice was the main concern has passed. Connecting enterprise data, keeping governance consistent, protecting sensitive information and controlling cost when usage spreads to thousands of people are now the actual decision items. Oracle's response is to run 50 to 70 dedicated regions worldwide along with Oracle Alloy, executing the same code across public cloud, customer dedicated facilities, Cloud@Customer, sovereign cloud and partner cloud. The meaning of multicloud has changed too. It used to be a defensive strategy for avoiding vendor lock in. Now, by Oracle's account, it is an offensive choice driven by the desire for "the freedom to pick the best of what is out there, wherever it sits."
In short, distribution is no longer the fallback you choose when the budget is tight. Enterprise requirements themselves have moved toward demanding it.
🤖 Signal 4. Inference has already moved down to the edge
So is it actually feasible to run inference in those scattered locations? The hardware side is answering.
NVIDIA's Jetson Orin Nano 2 delivers 78 TOPS of AI compute. Compared with the previous generation of the same size, inference performance is up to twice as high, and in 15W mode it holds the previous level of performance while using 40% less power.
What matters more than the performance is the tier. This product sits at the entry level of NVIDIA's lineup. It is not something that goes into a data center. It goes inside drones, cleaning robots and factory inspection equipment. Wing, an Alphabet subsidiary, is evaluating it for drone delivery, and Matic Robotics is putting conversational AI and gesture recognition into home cleaning robots. Companies handling quality inspection and anomaly detection on industrial sites, such as Cognex and Doosan Bobcat, are evaluating it as well. Until recently this level of inference required higher tier equipment or a cloud connection. That it now runs on an entry level module means inference is no longer something that happens only where special resources are available. The basis for dividing workloads changes with it. The first question is not how large the model is but where this computation should happen. The moment location becomes the criterion, infrastructure stops being one block and becomes a combination of several points.
🧭 The four signals in one sentence: training concentrates, inference scatters
Lay the four signals on top of each other and one picture emerges. Demand is growing in units of work executed rather than software licenses (Signal 1). As a result spending moves from applications to infrastructure, and inference becomes the largest operating cost (Signal 2). But that inference has to happen where the data is, and data is scattered by nature (Signal 3). And the hardware to run inference in those scattered places is already arriving (Signal 4). Large scale pretraining still requires massive concentrated infrastructure. That direction is not changing. What changes is inference. Inference has different requirements from training. Instead of running long in one concentrated batch, it arrives continuously, from many points, with loads that are hard to predict.
Category | Centralized premise | Distributed premise |
Primary workload | Large scale training | Continuous inference |
Cost structure | Based on reserved capacity | Based on actual usage |
Placement criteria | Region selection | Data location and latency |
Scaling method | Adding instances | Flexible node level scaling |
Key constraint | Available capacity | Unit cost and portability |
✅ What to check right now
If your team is running or preparing an AI product, these five checks are worth doing.
1. Recalculate inference cost in units of execution. You need a figure for cost per request and cost per user, not a monthly instance bill. Without that number you cannot predict what happens to margin as usage grows.
2. Sort your workloads into three groups. Large scale training, continuous inference and edge inference each need different infrastructure. If one setup is handling all three, cost is probably leaking somewhere.
3. Match model size to the workload. For repetitive tasks such as summarization, classification and structured output generation, a small model is often enough. Defaulting to the largest model is what pushes inference cost up.
4. Secure portability in advance. Being able to move infrastructure is what gives you negotiating power. It helps to check whether you have options that lower the cost of moving, such as an OpenAI compatible interface or container standards.
5. Map your data location constraints first. If you have not documented which data cannot leave which location, you will end up reversing decisions after the infrastructure design is finished.
✅ Distributed cloud: why Air Cloud is worth using
What matters in the end is lowering inference cost while still being able to arrange resources around where the workload and the data sit, and to move to different infrastructure when needed. Air Cloud addresses these conditions with a distributed infrastructure design.
Inference cost. Air Cloud connects GPU resources that already exist across multiple regions into a single infrastructure. Because it carries less fixed cost than building a new large scale data center, AI workloads can run at up to 40% lower cost than on a conventional centralized cloud. As inference volume grows, that difference in unit price feeds directly into product profitability.
Usage based scaling. Autoscaling is driven by actual token throughput rather than GPU utilization. Because the basis is what was processed rather than what was reserved, it moves with inference workloads that have irregular load.
Running where the data is. Private Air Cloud connects the GPU, NPU and CPU resources a company already owns into a dedicated cluster. Hybrid Air Cloud binds internal infrastructure and Air Cloud into a single cluster, expanding outward when traffic rises and failing over automatically during an outage. It sends computation to the data instead of moving the data.
Lowering the cost of moving. Air API provides an OpenAI compatible REST interface. Air Container deploys workloads on standard containers. You can swap an endpoint or migrate a container without major changes to an existing application, which makes it easy to compare cost and performance across infrastructures.
Establishing trust across scattered nodes. In distributed infrastructure, whether each node can be trusted matters more than how many resources there are. AIEEV combines external idle resources with nodes it operates itself, and applies an information security system based on ISO 27001 certification. Each node is verified with a TPM based hardware bound key, and communication between nodes is protected by an encrypted mesh network.
Air Cloud is not about distributing every computation unconditionally. Work that benefits from centralized infrastructure, such as large scale training, stays where it is. The point is to widen the options so that inference workloads sensitive to cost, data location and latency can run in a more suitable place. Put the model you are using now on Air Cloud and compare inference cost and response time.
