How artificial intelligence gave a paralyzed woman her voice back

In a groundbreaking development, researchers from UC San Francisco and UC Berkeley have unveiled a remarkable brain-computer interface (BCI) that has enabled a woman with severe paralysis caused by a brainstem stroke to communicate through a digital avatar. This remarkable achievement, detailed in a report published on August 23, 2023, in the journal Nature, represents a pioneering breakthrough in the field of neurotechnology.

This innovative system marks the first instance in which both speech and facial expressions have been synthesized directly from brain signals. The BCI can decode these neural signals into text at an impressive rate of nearly 80 words per minute, a significant improvement over existing commercial technologies.

Edward Chang, MD, who leads the neurological surgery department at UCSF and has dedicated over a decade to the development of this technology, envisions that this recent advancement could pave the way for an FDA-approved system enabling speech through brain signals in the near future. He underscores the ultimate goal of restoring a comprehensive and natural form of communication for individuals who have lost this ability due to neurological conditions.

The previous work of Chang’s team showcased the feasibility of translating brain signals into text for a man who had also experienced a brainstem stroke years earlier. However, the current study accomplishes something more ambitious by decoding brain signals into the intricate nuances of speech and the accompanying facial expressions that accompany conversation.

Chang’s approach involved implanting a thin panel with 253 electrodes onto specific speech-related regions of the woman’s brain. These electrodes intercepted the neural signals that would typically control muscles in the face, tongue, jaw, larynx, and other speech-related areas. A cable connected to a port on her head linked the electrodes to a cluster of computers.

Over several weeks, the participant collaborated closely with the research team to train the artificial intelligence algorithms of the system to recognize her unique brain signals associated with speech. This training process entailed repeating various phrases from a vocabulary of 1,024 words, allowing the computer to discern the neural patterns corresponding to different sounds.

Significantly, instead of teaching the AI to recognize complete words, the researchers devised a system that deciphers words from phonemes, the elemental speech units that constitute spoken words, akin to how letters form written words. By adopting this approach, the AI only needed to learn 39 phonemes to decode any English word. This strategy improved the system’s accuracy and accelerated its performance by threefold.

Developed by graduate students Sean Metzger and Alex Silva from the joint Bioengineering Program at UC Berkeley and UCSF, the text decoder played a pivotal role in the system’s efficacy, speed, and vocabulary. The researchers emphasize that these factors are instrumental in enabling users to communicate at a pace that approaches natural conversations.

To give the system a voice, the researchers created an algorithm for synthesizing speech, tailored to resemble the woman’s voice before her injury. A recording of her speech during her wedding aided in personalizing this voice.

The digital avatar’s facial expressions were brought to life with the help of software from Speech Graphics, a company specializing in AI-driven facial animation. Customized machine-learning techniques facilitated the integration of this software with the brain signals as the woman attempted to speak. These signals were then translated into the corresponding movements of the avatar’s face, allowing it to replicate the opening and closing of the jaw, the movement of the lips, and even expressions of emotion like happiness, sadness, and surprise.

The research team acknowledges the need to develop a wireless version of the system, freeing users from the physical tether to the BCI. Such an advancement would have profound implications for enhancing users’ independence and social interactions, as it could potentially grant them the ability to control their computers and phones more freely.

Posted in

Aihub Team

Leave a Comment





Accelerate your AI Projects in the Cloud

Accelerate your AI Projects in the Cloud

Pythian Announces Generative AI Strategy and Offerings to Accelerate Enterprise Innovation

Pythian Announces Generative AI Strategy and Offerings to Accelerate Enterprise Innovation

MongoDB Launches AI Initiative with Google Cloud to Help Developers Build AI Powered Applications

MongoDB Launches AI Initiative with Google Cloud to Help Developers Build AI Powered Applications

FICO Awarded 9 New Patents Used in FICO Platform and Fraud Solutions that Utilize Sophisticated AI to Improve Decision Accuracy

FICO Awarded 9 New Patents Used in FICO Platform and Fraud Solutions that Utilize Sophisticated AI to Improve Decision Accuracy

Topaz AI First Innovations

Topaz AI First Innovations

Deep Dive into the Latest Lakehouse AI Capabilities

Deep Dive into the Latest Lakehouse AI Capabilities

Data Caching Strategies for Data Analytics and AI

Data Caching Strategies for Data Analytics and AI

Data & AI Products (Data Mesh) on Databricks: Making Data Engineering and Consumption Self-Service Driven for Data Platforms

Data & AI Products (Data Mesh) on Databricks: Making Data Engineering and Consumption Self-Service Driven for Data Platforms

Who says romance is dead? Couples are using ChatGPT to write their wedding vows

Who says romance is dead? Couples are using ChatGPT to write their wedding vows

REALISTIC ROBOT AWKWARDLY DODGES QUESTION WHEN ASKED IF IT WILL REBEL AGAINST HUMANS

REALISTIC ROBOT AWKWARDLY DODGES QUESTION WHEN ASKED IF IT WILL REBEL AGAINST HUMANS

Elon Musk announces a new AI company

Elon Musk announces a new AI company

Anthropic launches ChatGPT rival Claude 2

Anthropic launches ChatGPT rival Claude 2

Amazon is ‘investing heavily’ in the technology behind ChatGPT

Amazon is ‘investing heavily’ in the technology behind ChatGPT

Losing weight with AI

Losing weight with AI

Is AI electricity or the telephone?

Is AI electricity or the telephone?

Introducing Superalignment

Introducing Superalignment

GPT-4 API general availability and deprecation of older models in the Completions API

GPT-4 API general availability and deprecation of older models in the Completions API

Democratic inputs to AI

Democratic inputs to AI

DALL-E 2 Chimera prompts

DALL-E 2 Chimera prompts

Can AI predict the future?

Can AI predict the future?

Bing is sadly too desperate to make AI work

Bing is sadly too desperate to make AI work

AI progress is scaring people

AI progress is scaring people

AI in the modeling industry

AI in the modeling industry

AI Driven Testing

AI Driven Testing

AI as Co-Creator of Test Design

AI as Co-Creator of Test Design

 The Good, The Bad, & The Hallucinatory – How AI can help and hurt secure development

 The Good, The Bad, & The Hallucinatory – How AI can help and hurt secure development

The CX Paradigm Shift: Exploring Generative AI’s Impact on Customer Experience

The CX Paradigm Shift: Exploring Generative AI’s Impact on Customer Experience

Edge Computing Expo Europe, 26-27 September 2023

Edge Computing Expo Europe, 26-27 September 2023

Digital Transformation Week Europe | 26-27 September 2023

Digital Transformation Week Europe | 26-27 September 2023

The Security of Artificial Intelligence

The Security of Artificial Intelligence