Brain implant lets paralysed patients speak via digital avatar

Scientists have harnessed brainwaves to enable people with paralysis to express words and gestures via a digital avatar, in the latest neurotechnology to help restore lost physical faculties.

The system is the first to decode speech and associated body movements simultaneously from one brain implant and then animate a virtual representation of a person, the researchers said.

The work is part of an expanding effort to use brain-computer interfaces (BCIs) to help people deprived of speech convert their thought patterns into conversations.

“We’re trying to move beyond BCIs that restore one function at a time towards a single brain interface that can restore multiple ways of communicating and moving,” said Samantha Brosler, lead author of a paper on the research published in Nature Neuroscience on Monday.

“Gestures can add meaning, emphasis, emotion and context, and in some cases can substitute for speech entirely — for example, nodding yes, shaking your head no, waving, shrugging or giving a thumbs-up.”

The proof-of-concept study by Brosler and colleagues at the University of California, San Francisco (UCSF) used a small set of communicative gestures: 10 in one participant and four in another. The system, boosted by machine learning, accurately decoded words and gestures from the thoughts of both people.

The team’s next goal is to “expand this approach to a much larger vocabulary and [a] more continuous range of movements across the body,” Brosler said.

Effective methods of communication prostheses could help millions of people who have full or near-intact cognitive function but have lost speech faculties due to a stroke, the neurodegenerative disorder ALS and other brain conditions.

In the longer term, scientists hope it might be useful to people who have difficulty vocalising because of cerebral palsy and autism.

The field of communication prostheses has attracted growing research interest and investment. Companies active include Elon Musk’s Neuralink and Precision Neuroscience. Echo Neurotechnologies, co-founded by Dr Edward Chang, principal investigator on the UCSF work, is developing brain-implant hardware.

The UCSF team’s latest system was innovative and offered new clues on how the brain combined speech with other forms of communication, said Richard Rosch, a brain dynamics specialist and senior clinical lecturer at King’s College London.

“The study shows significant overlap between where in the brain speech and hand gestures are processed,” Rosch said. “[This] adds to an emerging literature that the human brain may not be a network of highly specialised, connected-but-separate regions, but appears to have a lot of overlap in what function individual brain regions assume.”

But Rosch warned the technology was far from being accessible. It required significant brain surgery and performed less well at processing movements on the other side of the body from the implant, he added.

Scott Wellington, a researcher at the Bath Institute for the Augmented Human at the University of Bath, praised the research but said obstacles remained to turning it into a standard assistive tool. These included that the system was bespoke for each patient.

“These techniques are largely subject-dependent — that is, the models trained on one person’s brain signals will work for that individual, but struggle to carry across to others,” Wellington said. “[Big] investments will be required to research generalisable models, which may preclude widespread clinical adoption.”