AI persistent memory systems face vulnerability to manipulation

Research led by Arulnidhi Karunanidhi has identified significant vulnerabilities in the persistent memory systems of AI agents. The study, conducted using the LongMemEval corpus, demonstrates how simple false statements can be used to manipulate an AI's memory without the need for sophisticated hidden instructions or specific triggers.

The findings show that poisoning just 1.2% of a corpus can cause retrieval accuracy to drop from 0.850 to 0.300. Current defense mechanisms, such as content-based controls and provenance-weighted retrieval, showed limited effectiveness in preventing these accuracy collapses or rejecting poisoned memories.