Xiaomi releases Xiaomi-TabLDM open-source tabular data AI model

Xiaomi has released Xiaomi-TabLDM, an open-source foundation model designed specifically for structured tabular data. Unlike traditional machine learning methods such as XGBoost or LightGBM, which typically require retraining and parameter tuning for every new dataset, TabLDM utilizes a pre-trained approach that allows for classification and regression tasks without the need for task-specific fine-tuning.

The model was pre-trained on large-scale synthetic data generated from structural causal models. It features an architecture incorporating dual-stream feature grouping, lightweight Attention Residual, and sparse Mixture-of-Experts to model feature relationships at various granularities. Additionally, the model explores Test-Time Scaling to improve prediction performance by increasing computation during the inference phase.

In benchmark testing, Xiaomi-TabLDM ranked first in the OpenML-CTR23 regression benchmark and placed in the top tier for TALENT, TabArena, and BCCO. Compared to Google’s TabFM, the model demonstrated an 82% reduction in training time and a 68% reduction in prediction time. The model is available via GitHub and Hugging Face with a scikit-learn compatible interface.