Google DeepMind releases EmbeddingGemma 2 multimodal model

Google DeepMind has released EmbeddingGemma 2, a lightweight, multimodal embedding model capable of mapping text, code, images, video, and audio into a single, unified vector space. The model is designed for efficiency and on-device use, with a modular architecture that allows users to load only the necessary encoders. For instance, a text-only version requires approximately 191MB of RAM, making it suitable for smartphones.

Researchers have demonstrated the model's versatility by adapting it for speech-to-text tasks. By using a small reader to map audio slices into the model's text space, they achieved transcription speeds of approximately one hour of audio in 12.6 seconds on an M3 MacBook Air. While the current word error rate is around 5%, developers aim to close the gap with leading speech models through further training. The model is released under an Apache 2.0 license, allowing for commercial use.