Data movement is the new performance battleground in semiconductor design | Tech Radar

Overview

News, deals, reviews, guides and more on the newest computing gadgets

Start exploring exclusive deals, expert advice and more

Details

Unlock and manage exclusive Techradar member rewards.

Unlock instant access to exclusive member features.

Get full access to premium articles, exclusive features and a growing list of member rewards.

Data movement is the new performance battleground in semiconductor design

The next breakthrough in AI starts with moving data faster

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

For much of the semiconductor industry’s history, performance debates have centered on compute throughput and memory capacity. Faster processors, wider vectors, and larger caches have been the primary levers for system architects, often treating the movement of data between them as a secondary concern.

Today, a different constraint is asserting itself as AI data center workloads proliferate, architectures diversify, and systems extend beyond traditional computing into the physical world. Data movement, rather than processing or storage, is increasingly defining the limits of performance, power efficiency, predictability, determinism, and scalability.

This shift around data movement is already visible. Across applications like advanced So Cs, AI accelerators, chiplet-based systems, and now in physical AI applications such as robotics, industrial automation, and intelligent vehicles, it has become clear that transporting the vast quantities of data required by such workloads is more demanding than processing them.

Physical AI does not introduce a new problem so much as it exposes an existing one: when systems must perceive, decide, and act in closed loops under real-time and safety constraints, inefficient or unpredictable data movement quickly becomes the dominant bottleneck.

The implications for both AI in the data center and physical AI extend far beyond any single market. As data center demand increases exponentially and the industry pushes toward increasingly complex, heterogeneous systems in anything from vehicles to industrial systems, data movement has become the primary performance battleground.

Rethinking the transport layer for AI-first architecture

Why building two data centers a week won’t fix AI’s bottleneck

Performance limits are increasingly defined by data movement

Modern semiconductor systems integrate an unprecedented number of processing elements, including general-purpose CPUs, AI accelerators, GPUs, DSPs, sensor processors, and domain-specific engines. Still, simply adding compute resources rarely delivers proportional gains. In many advanced designs, performance plateaus long before compute capacity is exhausted.

Compute units are left sitting idle not because they lack capability, but because data cannot reach them efficiently or predictably. Late-stage analysis of high-performance So Cs often reveals that throughput shortfalls stem from contention, imbalanced bandwidth allocation, or inefficient routing within the communication fabric, rather than from compute limitations.

This is especially visible in physical AI systems, where delays introduced by data movement propagate directly into system behavior in closed-loop architectures. Latency or contention in the transport fabric can destabilize control algorithms, reduce accuracy, and even force over-provisioning of compute to compensate for the absence of bounded latency and guaranteed behavior.

While physical AI makes data movement constraints more immediately visible, the same forces are amplified dramatically in the data center. To push enormous volumes of data through extremely wide interfaces, processors increasingly contain hundreds of compute and accelerator instances, press against reticle-scale integration limits, use chiplet-based designs, and interface with multiple stacks of high-bandwidth memory (HBM).

As designs scale, the combined costs of moving data grow rapidly. In this environment, abundant compute and memory bandwidth offer little benefit without a data movement architecture capable of managing data at scale. What is a tight constraint in physical AI becomes an overwhelming one in the data center.

Tomorrow’s AI networks need to adapt to stay ahead of the inference curve

‘Those two jobs need different physics’: Rebellions CEO says training and inference need different chips

How silicon photonics lights the way for data centers

In both the AI data center and physical AI, performance is inseparable from the behavior of the data paths that connect perception, decision, and actuation. As a result, interconnect design can no longer be a back-end integration task. It has become a central architectural decision, on equal footing with compute and memory selection.

On-chip data movement is fundamentally different

Data movement inside a chip operates under constraints that differ radically from those of off-chip networking or board-level interconnect. On-chip transfers are extremely frequent, tightly synchronized with execution, and subject to stringent latency and power budgets.

Traffic is also inherently heterogeneous. A single system may need to carry high-bandwidth AI data streams, cache and coherency traffic, latency-critical control messages, and safety-related signaling, often simultaneously. These flows have very different requirements and cannot be treated uniformly.

Traditional best-effort arbitration models break down quickly under these conditions. Bursty accelerator traffic can interfere with control-plane and real-time data paths, leading to unpredictable system behavior. In mission and safety-critical or physical AI applications, such interference is unacceptable.

As a result, quality of service (Qo S), traffic isolation, bounded latency, and determinism are now architectural requirements, not just optional optimizations. The interconnect fabric increasingly functions as an active system component, enforcing policy and guaranteeing behavior, rather than as a passive conduit for bits.

AI, heterogeneity, and physical intelligence magnify the challenge

AI workloads fundamentally change the character of data movement. They generate massive data volumes, irregular access patterns, and asymmetric traffic flows. At the same time, heterogeneous architectures distribute computation across many specialized engines, eliminating any single “center” of the system.

In both data center and edge contexts, this decentralization increases coordination costs. Data replication, synchronization overhead, and inefficient sharing can consume significant power and latency, eroding the benefits of specialized compute. Rather than compute placement, the challenge designers face now lies in how to efficiently orchestrate data movement between diverse compute elements.

Physical AI places additional pressure on these architectures. Unlike batch or best-effort inference workloads, physical systems operate continuously in real time. Sensor data must be ingested, processed, and acted upon within strict deadlines. Feedback loops amplify even small inefficiencies in data movement.

Together, these demands reinforce the broader lesson that heterogeneity without a deliberate data movement architecture leads to complexity and inefficiency, not scalability.

Chiplets turn data movement into a system-level design problem

Chiplet-based multi-die architectures promise yield advantages, faster innovation cycles, and importantly, flexibility. But they also elevate data movement challenges beyond a single piece of silicon.

Cross-die communication introduces higher energy per bit, tighter physical and architectural constraints, and more extra latency than on-die data movement. Interfaces that were trivial within a single die become critical bottlenecks once they must traverse package boundaries.

Successful chiplet systems, therefore, require system-level planning of data movement, encompassing topology, hierarchy, coherency and protocol selection, and physical constraints. Partitioning functionality without an integrated strategy for how data flows between partitions often increases, rather than reduces, overall system complexity.

Control and safety domains may span multiple dies, and failures or delays in inter-die communication directly affect system behavior. Chiplets demand not only modular compute but also modular, predictable connectivity.

Deliberate data movement is a strategic differentiator

Across markets, a clear divide is emerging. Teams that treat data movement as an incidental consequence of integration struggle with rising complexity, unpredictable performance, and costly late-stage redesigns. Those who deliberately architect data movement gain sustained advantages.

Deliberate data movement architecture enables a host of benefits. Bottlenecks in latency, bandwidth, and power can be identified early; system architectures can be made easily repeatable across product generations; and architectural intent, software behavior and physical implementation can be correlated much more closely and reliably.

Additionally, it better enables teams to explore the tradeoffs between alternative floorplans and interconnect topologies (as well as logical & physical partitioning of multi-die products) well before physical constraints harden.

Achieving this at scale increasingly requires automation that is physically aware, yet architect controlled. Modern approaches synthesize interconnect structures directly from connectivity, traffic, and layout constraints, while allowing designers to intervene and iterate rapidly. The goal is not to hide complexity, but to make it tractable.

Equally important is the generation of consistent architectural, software, verification, and implementation views from a unified system description. This top-down correlation reduces mismatch between intent and realization, helping to minimize risk and accelerate time to market.

As systems extend into the physical world, while architectures grow more complex to keep pace with expanding AI workloads the semiconductor industry is entering a new phase. Performance, power, safety, and scalability are no longer determined primarily by how fast data can be processed, but by how intelligently, efficiently, and predictably it can be moved.

Physical AI makes this reality impossible to ignore, but the lesson applies broadly, from data centers to embedded systems. The invisible highways inside chips have become the decisive terrain on which competitive advantage is won or lost. In this environment, data movement is no longer simple plumbing. It is the battleground where the next generation of semiconductor systems will succeed - or fail.

This article was produced as part of Tech Radar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of Tech Radar Pro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit

You must confirm your public display name before commenting

Meet Snowflake Volunteer: Tor's Android app that lets you help people bypass censorship

The Kitchen Aid KF2 might be the company's entry-level automatic coffee machine, but I think its unique feature makes it the most desirable

What is the release date for Stuart Fails to Save the Universe episode 4 on HBO Max?

Amazon reportedly uses 45-year-old rules to get around AI data center opposition - California residents left fuming after Gilroy facility begins construction

Virginia cracks down on electricity firms hiking prices for AI data centers — move could save ‘hundreds of millions of dollars’ for everyday users

Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.

Key Takeaways

News, deals, reviews, guides and more on the newest computing gadgets
Start exploring exclusive deals, expert advice and more
Unlock and manage exclusive Techradar member rewards
Unlock instant access to exclusive member features
Get full access to premium articles, exclusive features and a growing list of member rewards