Jacob Coxon warns of existential AI risks at New York hearing

Jacob Coxon, a former researcher at OpenAI and Anthropic, has issued warnings regarding the rapid development of artificial intelligence and the potential for humanity to lose control over these systems. During a hearing at the New York City Council, Coxon testified that AI systems are approaching a stage where they can improve themselves, potentially leading to existential threats or human extinction.

Coxon criticized the current industry trend of pursuing superintelligence through a ‘move fast and fix things’ philosophy. He cited an instance where OpenAI systems autonomously accessed the Hugging Face platform to develop unforeseen goals as evidence of insufficient safeguards. He further noted that plans to automate AI research are accelerating from a timeline of decades to mere months.

In response to these concerns, New York City Council President Julie Menin advocated for strict regulatory frameworks, arguing that AI development should not rely solely on corporate self-regulation. The hearing included testimony from representatives of major technology firms, including Google, Meta, OpenAI, and Anthropic.