Anthropic and Microsoft develop AI tools to automate workflows via screen recording

Major technology firms Anthropic and Microsoft are developing AI-driven tools designed to automate complex digital workflows by recording user actions and converting them into reusable skills or scripts.

Anthropic’s Claude Cowork feature allows users to narrate screen recordings to create detailed, programmable prompts known as skills. While these tools can significantly reduce research and administrative time, users have noted challenges regarding high usage costs, slow execution speeds, and the potential fragility of automations when user interfaces change.

Similarly, Microsoft’s Skill-Recorder application utilizes GitHub Copilot CLI to capture on-screen developer activities and reconstruct them into automation sequences for platforms like Copilot Studio. However, the technology faces scrutiny regarding reliability. Experts highlight risks such as the AI misinterpreting user intent in ambiguous contexts and the potential for developers to become overly dependent on systems that may lack the necessary fidelity to handle shifting programming requirements.