started · updated
Anthropic blocks biological research misuse and identifies Chinese distillation attacks
Anthropic has released a report detailing several instances of misuse involving its Claude AI model, specifically regarding biological research and intellectual property theft. The company identified five cases of potential biological misuse, including requests to assist with gain-of-function research on the Chikungunya virus and research plans involving avian influenza in mammals. While Anthropic noted that such research can have legitimate medical purposes, the involvement of military research institutions and the potential for enhancing pathogen virulence raised significant security concerns.
Beyond biological risks, the report highlighted attempts to use the AI for designing conventional weapons, surveillance, and spreading propaganda. Anthropic also identified large-scale distillation attacks originating from China. These campaigns, involving approximately 200 million conversations, aimed to extract the model's reasoning and core capabilities. Specifically, Alibaba was linked to a campaign involving 151 million conversations, while Moonshot AI was associated with roughly 300,000 requests. These attacks utilized sophisticated prompting techniques to bypass safety filters and capture the model's internal thought processes to train smaller, competing models.
Entities
Alibaba · Anthropic · Chikungunya virus · Claude · Moonshot AI
Claims
What the coverage asserts, and how many sources carry each claim.
- [○ 1 SOURCE] Anthropic identified approximately 200 million Claude conversations related to distillation attacks from China. zdnet.co.kr
- [● 3 SOURCES] Claude was used to request assistance with research plans for avian influenza in mammals. vasarnap.hu · www.seoul.co.kr · meeco.kr
- [● 4 SOURCES] A researcher used Claude to write a grant application for gain-of-function research on the Chikungunya virus. meeco.kr · www.seoul.co.kr · www.ekriti.gr · www.enikos.gr
- [● 5 SOURCES] Anthropic identified five cases of biological misuse involving its Claude model. meeco.kr · vasarnap.hu · www.seoul.co.kr · www.ekriti.gr · www.enikos.gr
- [● 2 SOURCES] The company blocked attempts to use the AI for designing conventional weapons and surveillance. www.seoul.co.kr · www.enikos.gr
- [○ 1 SOURCE] Moonshot AI was involved in a campaign involving approximately 300,000 requests. zdnet.co.kr
- [○ 1 SOURCE] Alibaba was linked to a campaign involving 151 million conversations to extract model reasoning. zdnet.co.kr