The hidden operational costs of agentic AI | Tech Radar

Overview

News, deals, reviews, guides and more on the newest computing gadgets

Start exploring exclusive deals, expert advice and more

Details

Unlock and manage exclusive Techradar member rewards.

Unlock instant access to exclusive member features.

Get full access to premium articles, exclusive features and a growing list of member rewards.

Agentic AI demands efficient, always-on infrastructure redesign

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

Enterprise AI demands a fundamentally different infrastructure than the interactive, query-driven AI popularized by Chat GPT, Gemini, and other copilots. Instead, agentic AI — systems that autonomously plan tasks, execute workflows, call APIs, and make decisions with minimal human oversight — will drive enterprise adoption.

This new paradigm necessitates a computing foundation built for sustained, scalable efficiency, which is precisely where modern CPUs excel.

Chief Evangelist and leader of product marketing at Ampere Computing.

Unlike prompt-driven paradigms, agentic systems are designed to act, not just respond. Ideally, agents use smaller model sizes and often multiple models that are each domain experts at tasks such as image analysis, language interpretation, and transcription, often integrated with specific enterprise data

Why Agentic AI demands business process re-engineering

Why businesses are shifting from cloud to on-prem amid the agent boom

By monitoring signal data, initiating processes, and coordinating decisions across business environments, AI agents will become the productivity powerhouses of next-generation digital services. As organizations deploy agents more widely, the implications extend beyond application design to full-stack architectural overhauls.

Agentic AI doesn't just slot into existing workloads. This always-on, autonomic paradigm creates persistent, background compute demand. As AI agents proliferate, infrastructure efficiency becomes the critical determinant for scaling AI productivity.

Enterprises planning agentic strategies must therefore evaluate not only model capability, but also design compute infrastructure for continuous, efficient autonomous activity at scale.

A single agentic workflow can involve multiple model calls, data retrieval, validation loops, and downstream integrations. This continuous consumption profile necessitates an elastic operational layer, akin to the cloud-native application infrastructure enterprises are familiar with, but still nascent when applied to AI workloads.

This shift places distinct demands across the AI computing stack, particularly at the processing level. Efficient resource utilization techniques for specialized computing elements like GPUs are still decades behind CPU orchestration technology.

For agentic AI, the underlying CPU architecture becomes paramount, acting as the foundation that orchestrates these complex, continuous workflows. Infrastructure optimized for long-term training must adapt to deliver sustained performance at significantly lower costs to support at-scale agentic operations.

As agentic deployments scale, infrastructure demand grows, often in non-linear ways. Automated decisions generate follow-up processes, and workflows branch into additional tasks. Systems designed to increase productivity inherently increase the compute required to sustain that productivity.

Breaking free from pilot purgatory. The strategies needed to scale agentic AI

Don't let AI enthusiasm lock you into outdated infrastructure

Why single-player AI is holding back the agentic enterprise

This multiplicative effect is easily underestimated in early deployments. At scale, autonomy drives higher model utilization even as use cases evolve to increase functionality and responsiveness to variables like human interaction, new data sources, and context expansion in reasoning.

Enterprises will be continuously challenged to balance new AI functionality, escalating infrastructure demand from autonomous systems, and cost containment to meet their productivity goals.

An efficient, predictable compute foundation, such as that provided by Ampere processors, is crucial for managing this exponential growth without spiraling costs.

Given these challenges, persistent agentic inference generates ongoing energy and capacity requirements, leading to significant cost control challenges. AI workloads already operate at higher power density than traditional enterprise applications, and agentic systems extend this demand across longer time horizons.

In markets where high electricity costs and data center capacity are structural considerations, this dynamic has immediate operational implications. The ability to scale autonomous AI becomes directly tied to how efficiently it can run.

Provisioning infrastructure for peak responsiveness adds further pressure. Systems sized for maximum demand often operate well below capacity during steady-state periods, creating utilization inefficiencies that compound over time.

In these environments, efficiency and workload alignment matter more than theoretical peak performance.

Autonomy is ultimately an infrastructure decision

The economics of agentic AI are defined less by model acquisition or training investment and more by the ongoing cost of sustained autonomous activity. Energy consumption, cooling requirements, utilization rates, and operational overhead become the dominant variables.

These are precisely the metrics where modern, energy-efficient CPU architectures deliver significant advantages, allowing enterprises to run more AI with less power and space.

As agentic systems move deeper into enterprise workflows, AI transitions from a discrete tool to an always-on operational function, akin to managing human headcount burden rates.

At that point, innovation alone is not enough. Organizations must be able to run autonomy continuously, predictably, and within sustainable cost envelopes to hit productivity goals.

Agentic AI will reshape enterprise productivity, but its long-term viability hinges on infrastructure specifically designed for sustained agentic inference tasks, rather than intermittent training, AI experimentation, or even encyclopedic World Model use cases.

Efficiency, more than raw capability, will determine which organizations successfully achieve productivity gains and transform their businesses for the AI age.

Finding the efficient, scalable compute foundation required for continuous agentic AI will empower enterprises to unlock the full potential of autonomous AI without hidden operational costs.

This article was produced as part of Tech Radar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of Tech Radar Pro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit

Chief Evangelist and leader of product marketing at Ampere Computing.

You must confirm your public display name before commenting

1 The best of both worlds for soundbars and loudspeakers? Canvas hi-fi's new bar is pushing the definition, by packing huge 8-inch drivers into a frame that fits below your TV — and the company boldly says it's 'the world's first audiophile soundbar'

2 Modern Warfare 4 devs describe game as 'the biggest' and 'most impressive' Call of Duty they've worked on

320% of European Bank jobs at risk due to AI replacement, Morgan Stanley says

4I changed Chat GPT’s personality to act more like Gemini — and suddenly it felt like a completely different AI

5'We want to make sure that it feels really good to move, but it also feels good for the person who wants to shoot you' — Call of Duty: Modern Warfare 4 developer on balancing the game's next-level movement system

Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.

Key Takeaways

News, deals, reviews, guides and more on the newest computing gadgets
Start exploring exclusive deals, expert advice and more
Unlock and manage exclusive Techradar member rewards
Unlock instant access to exclusive member features
Get full access to premium articles, exclusive features and a growing list of member rewards