Groq to integrate 3D-stacked DRAM in next-generation AI chips

Groq plans to integrate 3D-stacked DRAM (3DS) into its upcoming Language Processing Unit (LPU) inference chips to address memory capacity limitations. While Groq’s current hardware relies on high-speed SRAM, which provides massive bandwidth but limited capacity, trillion-parameter AI models require significantly more memory to hold model weights.

The proposed design utilizes through-silicon vias (TSVs) to connect multiple DRAM layers vertically. This approach aims to increase memory density and capacity for large-scale AI models without sacrificing the high throughput necessary for efficient token generation. This shift follows a December 2025 deal in which Nvidia licensed Groq’s intellectual property and hired key personnel.