Part 1 looked at why AI agents are a 'program-shaped workload' that breaks the assumptions of existing serving systems, and at how the world's top systems conferences are approaching the problem. As the CacheBlend case showed, much of the answer depends on resources outside GPU memory, namely the DRAM and SSD that consumer PCs already have. Let's return to the cost question and continue from there, looking at why that direction converges on the picture of a "consumer GPU clou

Hello. I'm Yongrae Cho, and I lead AI infrastructure research and development on AIEEV's engineering team. I hold a PhD in Computer Science focused on the scalability and reliability of distributed systems, and I now work on the efficiency and reliability of LLM serving on Air Cloud, along with research and development in AI agent technology. As more services, like AI agents, call a model dozens or even hundreds of times for a single request, the inference bill grows far fast