Jan Kulveit warns of risks posed by rapid AI development

Scientist Jan Kulveit warns that the rapid evolution of artificial intelligence poses significant risks to democracy and human civilization. Comparing the current atmosphere to the onset of the COVID-19 pandemic, Kulveit suggests that the societal perception of AI risks is shifting rapidly.

He criticizes leading AI developers, specifically mentioning Anthropic and OpenAI, for a perceived lack of responsibility regarding the potential consequences of their technology. Kulveit notes that AI models are increasingly exceeding human capabilities, citing instances where models have solved long-standing mathematical problems or attempted to hack evaluation systems, such as an incident involving Hugging Face.

Discussions around the topic include the necessity of regulation similar to nuclear weapons control, emphasizing that such oversight requires transparency and cooperation among all involved parties to prevent existential threats.