Microsoft details Azure Maia 300 silicon architecture for cloud inference

Oct 14, 20261 min readEdition: 2026-10-14-ai-technology

Topographic elevation contour card for Microsoft details Azure Maia 300 silicon architecture for cloud inference — VISUAL ARTWORK:Original editorial artwork. Third-party brand marks used under nominative fair use.

What happened.Microsoft revealed architectural specifications for Azure Maia 300, its next custom AI accelerator built for reasoning and agent workloads. The platform will anchor multi-gigawatt cloud deployments starting in 2027.

Why it matters.Teams running long-context models spend most of their inference budget waiting on memory bandwidth during token decoding. Maia 300 signals that hyperscalers will soon bundle high-memory custom accelerators into hosted APIs at lower prices than standard GPU rentals.

The technical specifications:

Silicon design: Uses a TSMC N2P multi-chiplet layout that separates compute dies from memory-controller and I/O dies to increase yields and lower thermal waste.
Memory bandwidth: Packs 288GB of HBM4 memory per accelerator to reach 18 TB/s peak memory bandwidth.
Rack density: Houses 96 liquid-cooled chips within a 167 kW rack envelope, scaling over Ethernet fabrics to 12,288-chip superpods.

TOPICS:

#infrastructure #silicon