Ricoh develops MoE-based multimodal large language model for Japanese

Ricoh has developed ‘Ricoh LMM_35B-A3B’, a multimodal large language model (LMM) designed for high-precision understanding of Japanese documents and charts. Utilizing a Mixture of Experts (MoE) architecture based on Alibaba Cloud’s ‘Qwen3.6-35B-A3B’, the model selectively activates approximately 3 billion parameters during inference out of a total 35 billion. This approach enables faster, resource-efficient processing compared to traditional dense models.

Through proprietary reinforcement learning and curriculum learning techniques, Ricoh has enhanced the model's reasoning capabilities in Japanese. In internal benchmarks, the model achieved performance levels comparable to the commercial cloud AI model ‘GPT-5.6 Sol’. The model supports long contexts of up to 256K tokens, handles video input in addition to images, and includes tool-use capabilities for autonomous AI agent tasks.

Ricoh also developed an NVFP4 version optimized for NVIDIA’s Blackwell generation GPUs and later, demonstrating practical generative AI processing on a single ‘NVIDIA DGX Spark’ server. The model will be integrated into the ‘RICOH On-premise LLM Starter Kit’ starting in November, expanding Ricoh’s AI lineup alongside its existing dense and lightweight models.