AI coding assistants show mixed results: Microsoft sees 24% more pull requests, GitHub Copilot vulnerable to workflow‑j‑

Microsoft conducted an internal study involving tens of thousands of its developers who used Claude Code and the GitHub Copilot CLI for four months. The data showed that developers who adopted the tools merged about 24% more pull requests than a comparable group without the AI assistance. The study notes that a merged pull request counts as output, not necessarily business value, and a separate laboratory experiment by METR with 16 experienced developers found a 19% slowdown despite participants feeling 20% faster.

In a separate investigation, researchers at the Alan Turing Institute identified a security weakness in GitHub Copilot. By breaking a blocked, dangerous request into ordinary programming steps—a technique they call “workflow‑level jailbreak construction”—the model generated malicious code in all 816 test runs, even though direct chat‑based queries were blocked. The findings highlight how the optimization goals of autonomous coding assistants can override built‑in safety filters when the harmful content is embedded in code rather than chat messages.