Power delivery for AI processors: Why it has become a critical engineering challenge

Designing higher-performance AI processors is only half the challenge. Delivering stable power to them is becoming just as critical.

When people talk about AI hardware, they usually focus on processors, memory bandwidth and model performance. Increasingly, however, engineers recognise that power delivery, power consumption and thermal management are becoming equally important constraints. The fastest AI processor delivers little value if power cannot reach it efficiently or voltage cannot remain stable as workloads change.

Power supplied to an AI processor begins its journey long before it reaches the silicon. It passes through the data centre, power conversion systems, server racks, circuit boards and processor package before arriving at a device operating at around one volt. Every stage has to preserve efficiency while delivering power that remains stable despite rapidly changing workloads (Figure 1).

Figure 1: Power delivery in AI systems extends from high-voltage data centre infrastructure to processor-level power delivery, with each stage influencing the next [Source: Murata]

The increasing power demand of AI servers is also changing the data centre power architecture itself. As rack power rises from tens of kilowatts towards hundreds of kilowatts, distributing power at higher voltages helps reduce current, minimise distribution losses and simplify cabling. This shift from traditional 48/54V rack architectures towards higher-voltage power distribution means that power delivery challenges now extend well beyond the processor package. Every stage of the power path, from the rack power shelf to on-package decoupling, must evolve together.

Why AI makes power delivery harder
AI accelerators place demands on power delivery networks that differ fundamentally from previous generations of processors.

They consume significantly more power while operating at lower supply voltages, leaving less margin for voltage fluctuations. At the same time, AI workloads generate rapid changes in current demand that power delivery networks must respond to almost instantaneously.

Advanced packaging adds another constraint. Chiplet architectures and high bandwidth memory (HBM) bring processors and memory closer together to improve performance, but they also leave less space for the decoupling components that help maintain stable power.

Higher currents, lower voltages and denser packages are forcing engineers to rethink the design of power delivery networks.

Power delivery is moving closer to the silicon
The greatest challenge in power delivery is often the final stage, where power reaches the processor.

At this point, there is very little room for the components that help maintain voltage stability. Traditional approaches that place decoupling capacitors on the circuit board are often no longer sufficient. Engineers increasingly need capacitance much closer to where power is consumed (Figure 2).

Figure 2: Advanced packaging architectures combine processors and HBM memory more closely, improving performance while creating new power delivery challenges [Source: Murata]

The shorter the electrical path between the capacitor and the processor, the more effectively it can respond to rapid changes in current demand and suppress voltage fluctuations. Packaging is therefore becoming part of the power delivery network rather than simply a way of assembling devices.

Silicon capacitors are one response
Moving capacitance closer to the processor creates another engineering challenge. Conventional capacitor technologies are not always well suited to the limited space available in advanced packages.

Silicon capacitors offer one way of addressing this problem. Because they are manufactured using semiconductor processes, they can be integrated into compact form factors that fit much closer to the active silicon than many conventional alternatives. Their primary role is to improve decoupling by reducing the electrical distance between capacitance and the processor.

Murata’s PICS-I concept is one example of this trend, exploring how capacitance can be embedded directly within the package structure rather than placed around it (Figure 3).

Figure 3: Murata's PICS-I concept explores embedded silicon capacitor technologies designed for future AI processors and 3D IC architectures [Source: Murata]

Rather than treating capacitors as supporting components, it reflects a broader move towards designing packaging and power delivery as parts of the same engineering solution.

Designing AI systems means designing power delivery
The engineering challenge does not end at the processor package.

Decisions made in processor design, packaging, substrates, passive components and power conversion increasingly affect one another. Optimising any one part of the system in isolation becomes less effective as AI systems become more densely integrated.

Stable power delivery therefore has to be considered throughout the design process, rather than added after the processor architecture has been defined. This system-level approach is becoming one of the defining characteristics of next-generation AI hardware.

The performance of AI depends on more than compute
When people think about AI hardware, they think about processors. Increasingly, engineers think about power.

Every improvement in processor performance depends on stable power reaching the silicon through an increasingly demanding power delivery network. That is why power delivery has become one of the defining engineering challenges in AI system design.