Microsoft unveiled Maia 300, its newest internally developed AI accelerator, designed for inference, SiliconANGLE's Maria Deutscher reported on October 12. Microsoft says the Maia series will become its primary source of inference capacity over time.
The hardware
Maia 300 has four computing chiplets built on TSMC's 2-nanometer node, flanked by eight HBM4 modules with 288 GB of combined capacity, the same as Nvidia's flagship Rubin accelerator, but with about 20% less memory bandwidth than Rubin. A device control chiplet manages power flow and collects telemetry, and two auxiliary chiplets manage data moving into and out of the accelerator. The computing chiplets keep information in internal SRAM cache when possible and use a physically nearby memory stack when HBM is unavoidable. Data is compressed into lightweight formats such as MXFP4.
Microsoft's claims
Microsoft estimates 40% more tokens per megawatt than leading GPU-based systems. SiliconANGLE says that likely refers to Nvidia products, which is the author's inference, not a comparison Microsoft stated. Microsoft expects Maia 300 racks to deliver about seven times the token throughput of a system with the previous-generation Maia 200. Two Microsoft executives, Rani Borkar and Saurabh Dighe, wrote in an Azure blog post that “Narrower representations reduce memory footprint and data movement,” enabling larger models, longer contexts and more concurrent sessions in the same power envelope.
Scale and networking
Trays hold four accelerators each and liquid-cooled racks hold 96. Microsoft plans to group racks into superpods of 12,288 accelerators, linked by a custom Ethernet-based network stack that can adjust bandwidth between in-rack and inter-rack traffic and uses the same communication semantics for both, removing the need for host CPUs or standalone network chips for switching. As they put it, jobs can use capacity across rack boundaries, reducing stranded accelerators.
Timing
Maia 200 debuted in January and is already in production. Microsoft plans to start deploying Maia 300 in production next year. The benchmarks here are Microsoft's, and the article does not cite independent testing.
Source: SiliconANGLE. This is an AI Business Watch summary of the original report.