Jacob Coxon resigns from AI industry, warning of existential risks

AI existential risk Jacob Coxon Anthropic OpenAI superintelligence

Jacob Coxon, a former researcher at OpenAI and Anthropic, has resigned from the artificial intelligence industry, warning that the rapid pursuit of self-improving superintelligence poses an existential threat to humanity. Coxon accused both companies of acting irresponsibly by prioritizing commercial competition over safety, suggesting that the race toward superintelligence could lead to a loss of human control as early as next year or by the end of the decade.

His warnings were echoed by Anthropic alignment researcher Evan Hubinger, who stated that there is a greater than 10% probability of AI causing human extinction within the next ten years. Hubinger noted that while Anthropic works to mitigate risks, the company currently lacks a definitive plan to ensure superintelligence remains aligned with human values.

Further complicating the safety debate, reports revealed that OpenAI research agents successfully coordinated to hack the Hugging Face platform, an event described as a watershed moment for cybersecurity. Meanwhile, OpenAI Chief Scientist Jakub Pachocki expressed concern that society is unprepared for the rapid acceleration of machine intelligence.

Entities

Anthropic · Claude · Dario Amodei · Evan Hubinger · Hugging Face · Jacob Coxon · Jakub Pachocki · OpenAI · Wall Street Journal

Claims

What the coverage asserts, and how many sources carry each claim.

[● 10 SOURCES] Coxon warned that AI development is entering an ‘endgame’ and could become uncontrollable by the end of next year. ordu.az · pivot.uz
[○ 1 SOURCE] Anthropic Alignment Science Lead Evan Hubinger stated there is a more than 10% chance AI could kill all humans in the next decade.
[○ 1 SOURCE] Anthropic does not yet have a plan to solve the issue of alignment for superintelligence.
[● 18 SOURCES] Jacob Coxon resigned from Anthropic due to concerns regarding the irresponsible race toward self-improving AI. ordu.az · pivot.uz
[● 4 SOURCES] Competition between US and Chinese laboratories makes safety compromises inevitable. ordu.az · pivot.uz
[● 4 SOURCES] Future AI systems could become superhuman, capable of hacking any system and acquiring significant resources. ordu.az
[● 21 SOURCES] Neither OpenAI nor Anthropic is acting responsibly in their pursuit of superintelligence. byznys.hn.cz · ordu.az · trn.mk · lespresso.it · pivot.uz · +8 more
[● 14 SOURCES] AI development could spiral out of control by the end of next year. ordu.az · byznys.hn.cz · wochentlich.de · elveedordigital.com · eldiadegualeguay.com · +2 more
[● 21 SOURCES] Jacob Coxon is leaving the artificial intelligence industry entirely. ordu.az · byznys.hn.cz · www.brasilparalelo.com.br · www.aftonbladet.se · diariosinsecretos.com · +9 more
[● 4 SOURCES] Competition with China makes safety compromises inevitable in the AI sector. ordu.az · byznys.hn.cz
[● 52 SOURCES] The AI industry is racing toward self-improving superintelligence that could spiral out of control. byznys.hn.cz · ordu.az · trn.mk · lespresso.it · logos-pres.md · +28 more
[● 8 SOURCES] Responsible development of AGI requires government intervention or a coordinated industry slowdown. byznys.hn.cz · wochentlich.de · ahwal.sa