AI shopping assistants struggle with accuracy in new study

A recent study by Product.ai indicates that artificial intelligence is not yet capable of fully managing holiday shopping for consumers. The research tested major Large Language Models (LLMs), including ChatGPT, Claude, Gemini, and Perplexity, across 220 different shopping scenarios involving various products like electronics and household goods.

The findings revealed that 86% of the queries resulted in “reproducible factual contradictions.” These discrepancies included inconsistent information regarding product prices, specifications, and model names. Dakota Nunley, head of search products at Product.ai, noted that LLMs have not yet reached a practical level for end-to-end shopping experiences due to their inability to provide the accurate foundational information required for purchasing decisions.