Nvidia developing Nemotron 4 AI model with one trillion parameters

Nvidia is reportedly developing Nemotron 4, a new generation of AI models that could include a version with at least one trillion parameters. This move represents a strategic expansion for the chipmaker as it seeks to compete more directly in the AI software and open-weight model space.

The largest anticipated model in the Nemotron 4 family would be twice the size of the Nemotron 3 Ultra. While a trillion-parameter scale is significant, reports indicate that Chinese laboratories have already reached higher capacities, such as Moonshot AI’s Kimi K3 with 2.8 trillion parameters and DeepSeek V4 Pro with 1.6 trillion parameters.

Nvidia has significantly increased its investment in this sector, reportedly tripling its cloud spending on in-house model training to $28 billion through 2031. Developing these models allows Nvidia to influence how its hardware and software are utilized, though it also places the company in potential competition with its own major customers, such as OpenAI.