DeepSeek and Huawei drive shift toward lightweight AI hardware and reduced CUDA dependency

The market for integrated large language model (LLM) hardware is undergoing a significant shift. Following the popularity of DeepSeek R1, many manufacturers released high-end integrated machines capable of running massive 671B parameter models. However, these high-end units have faced challenges including high costs, limited concurrency, and difficulty in scaling via cluster deployment. Consequently, the market is pivoting toward lightweight, cost-effective AI BOX devices and smaller models (such as 32B or 70B parameters) that better suit common tasks like document writing and knowledge base queries.

Simultaneously, efforts to reduce reliance on NVIDIA's CUDA ecosystem are accelerating. DeepSeek has open-sourced DeepGEMM-Ascend, an AI foundation software designed to accelerate matrix operations specifically for Huawei's Ascend AI accelerators. This software is API-compatible with the original DeepGEMM, allowing developers to use existing workflows on Ascend hardware. Huawei has collaborated on this development, including optimizing supernodes utilizing 128 Ascend 950 chips. This move aims to bridge the performance gap between different hardware architectures and provide more robust alternatives to traditional GPU-centric AI deployment.