started · updated
Microsoft unveils Azure Maia 300 AI chip to reduce inference costs
Microsoft is advancing its internal semiconductor development to reduce the costs of its Azure AI services. By designing proprietary hardware, the company aims to lower inference costs per query and stabilize margins by reducing reliance on expensive third-party GPUs.
At the OCP Global Summit, Microsoft introduced the Azure Maia 300, the third generation of its internal AI accelerator, with a planned rollout in 2027. Built using TSMC’s N2P process, the chip is designed to optimize compute logic and memory connectivity specifically for AI workloads.
The Maia 300 features a power consumption of 950 watts and includes eight HBM4 memory stacks providing 288 GB of capacity and 18 TB/s of bandwidth. When bundled into racks of 96 accelerators, the system is intended to achieve up to a sevenfold increase in token throughput per rack compared to the previous Maia 200 generation. Microsoft claims the system will also deliver approximately 40 percent more tokens per megawatt than leading GPU systems, though these performance metrics have not yet been independently verified.