UCSF researchers develop brain implant to decode speech and gestures

Researchers at the University of California, San Francisco (UCSF) have developed an experimental brain-computer interface (BCI) capable of simultaneously decoding speech and physical gestures. Published in the journal Nature Neuroscience, the study demonstrates that a single implant can translate neural activity from the sensorimotor cortex into commands for a personalized virtual avatar.

Unlike previous BCIs that addressed speech or movement in isolation, this system allows patients with severe paralysis—resulting from conditions such as ALS or stroke—to communicate more naturally. By using machine learning to interpret brain signals, the device enables the avatar to replicate facial expressions and manual gestures, such as nodding, shrugging, or fist pumps, alongside spoken words.

In tests involving three patients, the technology successfully drove full-body avatars. One participant achieved 100% accuracy in decoding both speech and gestures during specific assessments, marking a significant step toward restoring more complex, multi-modal communication for individuals with motor impairments.