Agentic AI is moving enterprises out of the pantry and into the kitchen

Enterprise AI has only really been around for a few years, but let’s be honest: it feels like much longer. That’s because AI is one of those rare technologies that really does rip up the rule book and change how businesses operate. It may be new, but many enterprises are already wondering how they got by without it.

We’re still only at the very start of the AI story. A major development is unfolding that will change the face of enterprise AI and demand a completely new architectural response. Businesses that adapt to this new requirement fastest will be able to optimise their AI workloads and realise a fuller return on their infrastructure investments ahead of the rest.

The shift at hand is the adoption of Agentic AI by enterprises. According to McKinsey, 40% of large organisations report scaling AI agents, up from 27% last year. IDC believes that nearly half of global enterprises will orchestrate networks of AI agents across core business functions by 2030. Enterprise Agentic AI is coming sooner than we think.

From cafeteria tray to private chef

Agentic AI is a whole different game from the Generative AI (GenAI) we’ve all grown used to over the past few years. The workloads run differently, and as a result, have very different architectural requirements.

GenAI workflows are relatively simple. A user enters a prompt into a chat browser, which is then forwarded to an LLM for processing. The response generated by the LLM is then returned to the user. It's a linear, closed back-and-forth between the user and the AI model. Think of it like a cafeteria. You go up to the desk, place your order, and then receive your tray of food from the server.

With agentic AI workloads, the starting point is the same: a user enters a prompt, but from there things rapidly diverge. An AI agent takes over the work, splitting the prompt into a number of steps before deciding what action to take. Depending on its plan of attack, the agent then calls multiple models, queries databases, connects with APIs and runs apps in agentic sandboxes. It checks permissions, validates outputs, and then loops through the whole process again.

This is an open, continuous, and circular process. Forget your cafeteria. This is more like placing an order with a private chef who designs a menu, orders groceries, and manages a line of station cooks and waiting staff to design, coordinate, cook, and serve up a delicious meal.

The GPU is the oven. The CPU runs the kitchen.

When it comes to system architecture, Agentic AI is an order of magnitude more complex than basic GenAI.

For GenAI workloads, the focus is primarily on Graphics Processing Unit (GPU) acceleration. GPUs are the foundation of LLMs, providing the massive, concurrent mathematic processing needed for GenAI workloads. Central Processing Units (CPUs) have a role to play in GenAI, but it’s a relatively minor one, acting as a head node for servers with four to eight GPUs and taking care of functions like scheduling and system management.

In Agentic AI systems, CPUs are every bit as important as GPUs. GPUs remain vital for AI reasoning and raw maths processing power, but CPUs now have a broader role to play across a number of key functions. These include:

Supporting AI agents: In Agentic systems, CPUs take care of agent orchestration tasks such as agent management and coordination. These comprise processes essential to agent execution such as task decomposition, state and context management, and guardrails and governance.

Preparing data pipelines: CPU-hosted vector databases can help continuously feed real-time context into Agentic systems, providing agents with the up-to-date information they need to produce accurate results and to ensure GPU fleets are not left idle.

Supporting AI sandboxes: CPUs can provide capacity for the isolated sandbox environments agents require to generate, test, and run untrusted code safely.

Bolstering core workloads for demand surges: Core cloud apps were built for people, not agents. In multi-agent systems several agents can call tools simultaneously. This can exceed the capacity of existing systems and act as a drag on performance levels.

As businesses look to scale up and operationalise Agentic AI, we’re therefore heading to an environment where GPU and CPU ratios are more balanced. According to AMD, whereas GenAI servers typically require four to eight GPUs for one CPU headnode, agentic systems could see a 1:1 ratio and, in some cases, the scales might even tip in favour of CPUs.

Prepare the kitchen before service starts

Enterprises therefore need to focus on their CPU strategy every bit as much as their GPU strategy. When investing in CPUs for agentic infrastructure, businesses should ask:

Are my CPUs of sufficient density to enable agentic AI deployment on top of the established back-end infrastructure?

Is the CPU core count high enough to support the quantities of agentic sandboxes multi-agent systems require?

Do the CPUs deliver the low latency needed to support agentic tool calls and external queries?

Can I easily and cost effectively scale my core cloud workload CPU capacity to support the Agentic AI demand surge?

Are my CPUs compatible with the software used across the stack, or do I need to go through costly and time consuming refactoring?

Enterprises should look for CPUs that enable them to answer “yes” to all these questions. Doing so, they will build their Agentic AI systems on the strongest of foundations. This in turn will allow them to rapidly operationanalise their AI deployments and start realising value ahead of slower moving competitors.