Strata enables DeepSeek-V4.1-Flash execution on consumer PCs

Developer Niko Veit has announced that the open-source AI inference engine Strata can now run the massive DeepSeek-V4.1-Flash model on consumer-grade hardware. By utilizing a quantization method called ‘GSQ-RCO’ developed by the ISTA-DASLab at the Austrian Institute of Technology, the software allows a model with approximately 763 billion parameters to operate using a combination of GPU VRAM, system RAM, and SSD storage.

Strata employs a ‘popular expert cache’ mechanism to manage the model's Mixture-of-Experts (MoE) architecture. This approach keeps frequently used expert modules in memory while storing the rest on an SSD, significantly lowering the hardware barrier. Testing indicates that a system with a 12GB GPU (such as an RTX 3060) and 46GB of RAM can achieve generation speeds of 8 to 10 tokens per second.

Three quantized versions are available, ranging from 1.75-bit to 2.25-bit precision, with download sizes between 239GB and 274GB. Despite the heavy compression, the 1.75-bit version reportedly maintains 98.7% of the original model's average performance in mathematics, knowledge QA, and programming benchmarks.