Google Cloud evaluates Gemma 4 performance on TPU hardware

Technical evaluations of Google’s Gemma 4 model family highlight performance metrics across various Cloud TPU generations and competing hardware. Testing conducted on Cloud TPU v5e and v6e using the vLLM framework shows that the Gemma 4 E2B model can achieve serving speeds of 144 tokens per second in bf16 and 243 tokens per second in int8 on a v5e chip.

Comparisons include various hardware architectures such as the NVIDIA L4 and Blackwell GPUs, AMD MI300X, and AWS Inferentia. The data indicates that a v6e-1 serves approximately 3.3 times the tokens of an L4 when using the same 26B checkpoint. The analysis focuses on key factors including native number formats, memory bandwidth, slice shape, and chip availability.

Google’s TPU roadmap includes the v5e, v6e (Trillium), v5p, and the Ironwood architecture, which was announced as a specialized chip for the inference age. Performance scales significantly across these generations, with Ironwood offering up to 4,614 TOPs and 192 GiB of HBM.