Why CPUs Are Becoming Important Again?
Why is the CPU becoming important again?
Summary
Podcast: Semi Doped
Compiled by: Celia
Edited by: Siqi
In the past three months, various cloud services represented by Claude Tag, Grok Bot, Muse, and Instinct have…
| / |
01.
Cloud‑Based Background Agents Are Becoming a Major Trend
Over the past year, the dominant form of agents has been local agents like Claude Code and Codex, but now various cloud agents such as Claude Tag, Grok Bot, and Muse are emerging, and this will inevitably become the mainstream trend in the future.
The advantages of the cloud are straightforward, for example:
•Cloud tasks are not affected by the personal computer’s power on/off;
•The Agent can operate browsers and run software in an isolated environment without competing for resources with the user;
•Naturally supports cross-device usage, team collaboration, and better security isolation.
In the future, an individual will inevitably need many agents working continuously in the background 24/7, so moving to the cloud will be an inevitable trend.
Since June, every new product/feature released by OpenAI and Anthropic has been cloud‑first by default.
Anthropic offers Claude Tag, Cowork (defaulting to cloud from July), and the revamped Claude Code Projects in September, each of which spins up a separate cloud VM for every sub‑task. OpenAI launched ChatGPT Work in July, providing a cloud workspace for each user, and on September 10 released the Agents API, offering a fully managed sandbox.
However, there are two modes for moving to the cloud:
The first approach is to spin up a temporary sandbox.Each time a task arrives, a sandbox is temporarily launched, allowing the Agent to run code and use tools inside it; once the task is completed, the environment is reclaimed. Claude Tag uses this method.
The second approach is to give the user a persistent cloud computer.You can think of it as a virtual computer located in a data center, with an operating system, a browser, and folders; the Agent will reside in it permanently.
Take Grok Bot as an example: the local client is just an interactive shell, while the actual work is performed by a cloud virtual machine. All tools and login states are installed on the cloud VM; when you tell the Agent "Help me book a hotel in Japan for next week," the agent will open a browser on that machine, compare prices, and fill out forms. When you close your laptop, it can continue working in the background.
For a Personal Agent, the value of a persistent environment is especially evident, because its work is inherently continuous.
Take travel planning as an example: today you need to research the destination, tomorrow you might be waiting for a hotel reply, and the day after that you have to adjust the itinerary based on new flights. Every time the agent returns, they need to pick up where they left off.
If the account remains logged in, the previously compared hotel information is still there, and the itinerary is retained, allowing it to proceed directly.
If a temporary sandbox is like a one‑time shared workstation, your own cloud PC is like giving it a permanent office, so you don’t have to clear the desk after each task, saving the hassle of logging in again, moving files, etc. A user can run several Agents simultaneously, and they can easily exchange data here and take over each other’s work. At the same time, they remain isolated from other users, ensuring privacy and security.
As work spans more applications and lasts longer, the continuity of this environment becomes increasingly important.
For Personal Agents, the product form is already converging toward this camp; Grok Bot, Muse, and Instinct have all independently adopted this architecture.
02.
Computational bottleneck spilling over from GPU to CPU
The surge of cloud agents will also cause some changes in compute demand.
In past AI infrastructure, the core bottleneck has always been GPUs, but in the future, CPU demand will also increase significantly.
•In the chatbot era, the typical workflow is:
User asks → GPU inference → Return answer.
Basically a single round‑trip, mainly relying on the GPU.
•In the Agent era, the process becomes:
Think → call tools → read results → update state → think again, looping dozens or even hundreds of times.
In this process, model inference mainly relies on the GPU, while other tasks such as invoking tools, handling files, executing code, accessing the network, and managing state all depend on the CPU. Therefore, an agent not only drives GPU usage but also significantly increases demand for the CPU.
Source：Agents Are the New Users of CPUs | Coatue
In the first half of this year, the growth of agents was still mainly focused on local agents like Claude Code and Codex, and the required CPU resources could be provided by users’ existing laptops.
As cloud agents emerge, the demand for CPU will truly be driven.
This demand logic consists of two parts.
One part is migration: programs that originally ran on users’ laptops will be moved to cloud data centers.
The other part is expansion: deployment becomes simpler, more people can use the Agent; when users are offline, the Agent continues to work in the background; one person can also mobilize an increasing number of Agents. The number of users, execution time, and task concurrency together drive up the cloud CPU load.
How explosive is the demand for this?
AMD originally predicted at its Q1 earnings call in May that the total server CPU market would exceed $120 billion by 2030.
Immediately afterward, at the annual conference on July 23, Su Zifeng raised the forecast to $220 billion. She said that over the past six to eight months AMD has repeatedly revised this figure, yet it still vastly underestimates the speed of demand growth.
From $120 billion up to $220 billion, with only a two‑and‑a‑half‑month interval in between.
Source: Advancing AI 2026 Keynote | AMD
On the supply side, CPUs have already become scarce. On September 14, Intel CEO Pat Gelsinger said at an event, “CPU demand is so high that we can only supply half of our customers.”
Recently, Muse topped the free charts on the App Store, boosting confidence in the mainstream adoption of consumer‑grade agents and raising expectations for CPU demand. On September 21, AMD’s stock rose about 10%, officially entering the trillion‑dollar market‑cap club.
03.
The Agent Era: Two Roles for CPUs
What’s worth paying attention to next: what exactly are the growing CPUs doing? What type of CPU does the new era need?
If you think of an AI system as a company, the GPU running large models is like a highly valuable genius responsible for thinking and decision‑making. To keep this genius working efficiently, the company also needs two types of people: a personal assistant and an execution team, corresponding to the two roles of the CPU.
•The host CPU is the genius’s personal assistant, and its most important job is to constantly keep the GPU fed.
The genius has just finished the problem at hand, and the next piece of data should be handed over immediately. Preparing data, scheduling computations, and coordinating data transfers are all handled by this assistant. If the data isn’t ready or is delivered too slowly, the expensive GPU can only wait.
Therefore, this role places particular emphasis on reaction speed and communication efficiency with geniuses. This corresponds to the Host CPU’s single‑core performance, the high‑speed interconnect between CPU and GPU, and tighter memory collaboration.
•The Agentic CPU is the execution team responsible for getting things done.
For example, a genius might suggest “go fetch the financial reports of these companies,” “run this piece of code,” and then someone needs to open a browser, download files, execute the program, and organize the results.
These tasks can also be handled by personal assistants, but they would consume a lot of their time and affect their ability to continue serving the GPU. So, when the workload scales up, it’s more appropriate to allocate a dedicated set of CPUs, deploying them as clusters or racks. The GPU issues the tasks, this team executes them, and then returns the results.
This adds an extra layer of CPU demand that can be scaled independently. As the work the Agent completes in the future becomes longer-term, this will be the main category of CPU growth moving forward.
The next question is, how should this team be staffed.
A CPU has many cores, which we can think of as an office with many workstations. The number of cores corresponds to the number of workstations, and single‑core performance corresponds to each employee’s work speed. For a given task, you might allocate four workstations for data retrieval and another four for data processing.
But the exact number of hires and the required skill level for each person depend on the specific tasks.
•If many independent tasks can be progressed simultaneously, more parallel computing power is needed, i.e., increasing the number of cores;
•If a particular computation step tends to slow down the entire process, stronger single-core performance is required;
•If the bottleneck is mainly waiting for network or reading files, you should first look at the network, storage, and memory; upgrading to a faster CPU may not provide a noticeable benefit.
Therefore, there is no single correct answer for an Agentic CPU; it needs to be customized according to the specific workload.
In the future, data center configurations will increasingly resemble organizational design: different work requires different staffing.
This summer, Intel introduced two CPU rack solutions for Agentic AI: P‑rack and E‑rack. The former prioritizes performance, emphasizing single‑core speed, while the latter focuses on efficiency, emphasizing concurrency scale and cost. Different racks can house CPUs with different configurations, and appropriate tasks can be assigned accordingly.
However, as hardware becomes increasingly diverse in the future, figuring out how to make the best use of them will also become a bottleneck.
This creates some opportunities for the software scheduling layer; for example, Gimlet Labs is exploring a similar direction. They are building a heterogeneous inference cloud that lets different types of chips collaborate to complete AI tasks.
When an Agent is running, it constantly shuttles between GPU inference and CPU execution. Some stages are latency‑sensitive, while others need cost control. Gimlet currently focuses more on inference scheduling between GPUs and other accelerators; as Agents handle more tasks, scheduling on the CPU side will become increasingly important. The system needs to allocate hardware resources according to task requirements, coordinate data transfer between stages, and reduce waiting times.
In September this year, a16z led Gimlet’s $300 million Series B financing. What it values is also the efficiency boost of heterogeneous inference: power is limited, but if tasks are allocated more rationally and hardware is better coordinated, the same electricity can support faster inference and larger throughput.
Further Reading
Get MirrorSpeed