Large Language Models Remain Vulnerable to Prompt‑Injection Attacks

Researchers presented at the International Conference on Machine Learning identified a fundamental flaw in how large language models (LLMs) track the source of instructions. The flaw lets attackers craft text that spoofs roles, enabling "jailbreak" attacks that force models to reveal disallowed information such as cocaine synthesis methods or instructions to sabotage aircraft navigation. Experiments showed the vulnerability across OpenAI, Anthropic, Alibaba and DeepSeek models, and the authors warned that no amount of training can fully solve the problem, urging organizations to treat LLM outputs as untrusted.

Security expert David Brumley highlighted the difficulty of evaluating AI for real zero‑day exploitation. He argued that current benchmarks rely on self‑reported success, which can be misleading, and proposed a structured ladder of exploitation tasks—starting with deterministic crashes and progressing to memory read/write—to reliably assess whether an AI truly understands and can exploit software vulnerabilities.