AI Cost Management: From Optional to Essential in Two Years
- Jul 8
- 7 min read
Ever left the air conditioner running all day while you were out?
You don't think about it when you walk into a cool apartment. It's only when the electricity bill arrives that you find out exactly what it cost. And staring at that bill, a familiar thought creeps in: "Did I really use that much?" š
Right now, infrastructure teams at companies everywhere are having the same thought while staring at their AI bills. The only difference is the number is a lot bigger. Two years ago, this was a concern for only a handful of forward-thinking teams. The share of FinOps teams that included AI spend in what they actively manage was just 31% in 2024. This year, it's 98%. Almost every organization has started scrutinizing this bill closely. What pulled everyone toward the bill this fast? And now that they're looking closely, do these organizations actually know how much they're spending? This piece follows both questions.
1. From a Third to Nearly Everyone in Two Years
According to the State of FinOps 2026 report, the share of FinOps teams that include AI spend in what they manage came in at 31% in 2024, 63% in 2025, and 98% in 2026.
Break down the growth pattern and the two jumps look different in kind. The rise between 2024 and 2025 (31% to 63%) lines up with generative AI features moving out of pilot mode and into shipped products, the point where token costs stop being an R&D line item and become part of cost of goods sold (COGS). From here on, AI costs cross the threshold where you can no longer round them down to zero.
The rise between 2025 and 2026 (63% to 98%) has a different character. This window overlaps with the spread of agentic AI. A single user request now triggers dozens of downstream calls, search, tool use, re-inference, so costs swing by unpredictable margins even at the same traffic volume. Spend you can't predict is much harder to leave off your management list. This 98% means most organizations that could once treat AI spend as a rounding error have crossed into territory where ignoring it throws off the entire budget. The remaining 2% simply haven't crossed that threshold yet.
2. FinOps Moves From Finance to Engineering
The same report also shows a shift in org structure. 78% of FinOps organizations now report to a CTO or CIO. This isn't a simple reporting-line adjustment. Decisions like which GPU generation to use, whether to quantize a model, batch size, and autoscaling policy are now cost decisions. Cost optimization can no longer live purely in procurement or accounting. Whoever understands the architecture needs to sit where cost decisions get made, and that's a reasonable way to read why FinOps has physically moved toward engineering.
The scope of what FinOps covers has also widened. SaaS (90%), software licensing (64%), private cloud (57%), and data centers (48%) have all become areas FinOps teams now watch.
It's no coincidence that all four areas grew at once. AI spend isn't a new line item that belongs to any single one of these categories, it's scattered across all four. The same AI workload shows up on a SaaS invoice if you're calling an external model API, on a private cloud invoice if you're renting your own GPUs, and on a data center invoice if you're running hardware you bought outright. One expense just changes shape across multiple bills, so a FinOps team watching only one area misses the true scale of AI spend. That's why the scope had to widen all at once.
3. Why GPUs Don't Fit the Old FinOps Playbook
State of FinOps 2026 diagnoses the current moment this way: the obvious waste has mostly been cleaned up, and what's left is a set of smaller, more complicated savings opportunities. It's worth being specific about the difference between "easy waste" and "complicated opportunity." Turning off idle instances, switching to reserved instances, cleaning up unused storage, these are static optimizations you fix once and move on. Check it once, and the state barely changes before the next check.
GPUs don't fit that mold. The same GPU cluster shifts its usage pattern second by second depending on time-of-day traffic, model version, and batching policy. It isn't a problem you solve once, it's a dynamic optimization problem that demands continuous observation and continuous tuning. The report classifies GPUs as a "complicated opportunity" not because they're merely harder, but because the nature of the problem is different.
The numbers back up why dynamic optimization matters. According to cloud cost management platform Finout, average GPU utilization in enterprise Kubernetes environments is just 5%. There are structural reasons utilization runs this low. Guaranteeing response latency means keeping a minimum number of instances running even when there's no traffic. Teams avoid scaling workers down to zero to dodge cold starts, and often keep spare capacity on hand for traffic spikes. Every one of these safeguards leaves GPUs "on but not working." Billing, meanwhile, runs on however long the GPU stayed on, regardless of utilization. A GPU left on all day that actually worked for about an hour still shows no waste on any invoice.
The tagging problem sits at a different layer. A VM is used by one team at a time, so a single tag closes the loop on attribution. GPUs aren't like that. With MIG (Multi-Instance GPU) or time-slicing, requests from multiple teams often take turns on the same physical GPU. At that point, there's no clean resource to attach a tag to, you can only find out who used how much by instrumenting at the request or token level. Most cost management tools don't have that capability.
Both problems trace back to the same root. Existing FinOps tools were built around static resources like VMs and storage. A resource like a GPU, whose state changes by the second and is shared across multiple parties, falls outside that assumption entirely.
4. The Industry Is Already Moving, Just Not in the Same Direction
AIEEV isn't the only one feeling this problem. Looking at how cloud providers are responding shows the problem itself has more than one layer.
On June 9, 2026, AWS released FinOps Agent in public preview. When a cost anomaly is detected, the AI agent correlates the change event logged in AWS CloudTrail with the cost shift to infer a root cause, and answers natural-language questions like "why did my cost go up last month."
Look closely at this approach and its character becomes clear. It's a reactive diagnostic solution: it infers a cause from event records stored in CloudTrail after the cost has already gone up. That's only possible because every API call gets logged, but that volume of records is too much for a person to read and connect to a cost change by hand, so an AI inference step was added in between. The data had already been sitting there, it was the layer connecting that data to a cost change in real time that was missing.
The GPU cloud market shows a completely different response. Providers are racing to slice billing into finer units. Some bill by the hour, others by the minute, and under hourly billing, a 90-second job still gets charged a full hour, a structural source of waste. This approach has its own limit. Splitting the billing unit from hours to minutes makes "how much you paid" more accurate, but it still doesn't tell you "why you spent that much." A precise receipt and a transparent usage record are two different things. Just as AWS pushed root-cause tracing into a reactive AI inference step, finer billing units are still only solving the problem at the level of the invoice.
5. AirCloud's Approach: Visible by Design, Not Traced After the Fact
Line these two cases up and a shared limitation shows up. The point where the problem occurs (why did the GPU get used this way) and the point where you look for it (the invoice, log inference) are separated. AirCloud puts both of these on the same screen.
The dashboard shows usage and transaction history at 10-minute intervals. That 10-minute window isn't arbitrary. Look at second-by-second data and there's too much noise to tell which changes matter. Look at daily data and, by the time you check, you've already forgotten which deploy or traffic event caused the cost to move. Ten minutes is the interval where a person can still connect what just happened to the resulting cost change.
Container-log drill-down builds on top of that. Once you spot the window where cost spiked, you can open the container logs from that exact moment and see why autoscaling kicked in. You can tell directly from the logs whether traffic actually surged, a restart loop kicked off, or a batch job happened to overlap. Unlike AWS FinOps Agent, which has an AI "infer" a cause from CloudTrail events, this structure lets a person "confirm" it directly at the point where it happened, the container log. No separate inference step required.
Token-based autoscaling works in the same direction. Because billing is based on actual tokens processed rather than how long the GPU stayed on, the utilization problem covered in Section 3 (paying for a GPU that's on but not working) never arises in the first place. Since throughput is tallied at the request and token level, there's no need to force a resource-level tag onto a MIG or time-sliced environment. Both problems from Section 3 aren't solved after the fact, they're designed out of the billing structure itself. This isn't the only way to solve it. But it shows that instead of tracing GPU costs after the fact, you can design visibility in from the start, and that's the point AirCloud makes.
Closing
AI cost management is already an irreversible trend. But look at the solutions that have emerged so far, and most of them are just getting better at reading the invoice or the logs after something has already gone wrong. The next step isn't a more precise invoice, it's putting the invoice and the execution log on the same timeline.
Is your organization tracing that 5% utilization after the fact, or watching it right now?
References
FinOps Foundation, 2026.06 | https://www.nops.io/blog/state-of-finops-2026/
FinOps Foundation, 2026.06 | https://data.finops.org/
Finout, 2026.07 | https://www.finout.io/blog/why-traditional-finops-fails-when-gpu-costs-enter-the-picture
AWS, 2026.06 | https://aws.amazon.com/blogs/aws-cloud-financial-management/aws-finops-agent-is-now-public-preview/
GMI Cloud, 2026 | https://www.gmicloud.ai/en/blog/runpod-lambda-coreweave-gmi-gpu-rental-pricing
gpu.fm, 2026 | https://www.gpu.fm/blog/cloud-gpu-providers-comparison-2026
AWS builder center, 2026 | https://builder.aws.com/content/2d3URriy1PsUqBWuHr7Jj0SVzRD/finops-without-the-walk-of-doom-meet-the-aws-finops-agent
