AI safety and security risks

The rapid advancement of artificial intelligence is creating significant challenges for both human professionals and organizational security. Experts tasked with securing AI systems, including those at the UK government’s AI Security Institute, are reporting high levels of burnout and psychological distress due to the intense pressure of testing unreleased models. Some researchers have described the task of ensuring safety as a ‘mission impossible’ as technology integrates into high-risk sectors like military operations.

Parallel to these human-centric concerns, the rise of employee-driven AI workflows is introducing technical vulnerabilities. The emergence of ‘wild code’—unmanaged automations and agents created by staff—is outpacing traditional security processes. These workflows present risks such as excessive data permissions, sensitive data exposure, and ‘orphaned access,’ where automated processes continue to run with active credentials after an employee has departed. Experts suggest that addressing these risks requires formal oversight, identity tracing, and clear ownership to prevent useful automations from becoming unmanaged operational dependencies.