Anthropic warns of illegal AI model distillation and security risks

Anthropic is raising alarms regarding the illegal use of model distillation, a process where developers use the outputs of one AI model to train a competing “student model.” The company alleges that some overseas entities are bypassing usage restrictions through a sophisticated illegal ecosystem involving the dark web, stolen credit cards, and compromised accounts.

Anthropic specifically identified Chinese AI firm Moonshot AI, alleging its Kimi K3 model utilized Claude for training. The company has also previously accused Moonshot, DeepSeek, MiniMax, and Alibaba of similar practices, including a large-scale “distillation attack” by Alibaba. While Anthropic stated it does not oppose competition or all forms of distillation, it emphasized that unauthorized large-scale extraction of model outputs constitutes intellectual property theft and poses national security risks.

Beyond commercial competition, Anthropic warned that if advanced AI capabilities are obtained without proper safety guardrails, they could be misused for high-risk activities such as biological weapons development or large-scale espionage. To address safety in physical applications, Anthropic also introduced the “Model Hardware Standard,” a framework designed to govern how AI agents interact with physical machinery, such as laboratory equipment and manufacturing robots, to prevent accidents or malicious misuse.