Scientists Develop Virtual Prosthetic Voice System That Creates Speech from Brain Signals

The virtual prosthetic voice system deciphers what the brain wants to say and generates understandable speech, without any muscle movement.

0
148
Virtual Prosthetic Voice System Creates Speech from Brain Signals

Many people are not able to communicate due to strokes or head injuries suffered in combat or accidents, while some even fail to speak due to neurodegenerative disorders such as amyotrophic lateral sclerosis (ALS).

In what could be a groundbreaking medical discovery, scientists have reported that they have created a virtual prosthetic voice system that deciphers the vocal intentions of the brain and translates them into understandable speech, without actually moving a single muscle.

The physicist, cosmologist, and author Stephen Hawking had to use a muscle in his cheek to type keyboard characters, allowing a computer to synthesize speech.

The neurologist and neuroscientist at the Mayo Clinic Dr. Anthony Ritaccio said, “It’s formidable work, and it moves us up another level toward restoring speech by decoding brain signals.”

Scientists have developed other virtual speech aids, which work by deciphering the brain signals responsible for identifying and distinguishing words and letters, the verbal presentations of speech. However, those aids lack the fluidity and speed of a natural speech.

The new voice system, which was described Wednesday in the journal Nature, decodes the motor commands of the brain, guiding vocal movements during a speech – such as tapping the tongue and narrowing the lips.

It was tested on people who can speak normally; however, it has not been tested in people with neurological conditions, which could make the decoding impossible or difficult.

For a new clinical study, scientists at the University of California, San Francisco, and U.C. Berkeley have recruited five people who were in the hospital for the evaluation of epilepsy surgery.

Many people who have a history of epilepsy respond poorly to medications and often opt to undergo brain surgery. Before the surgery, doctors have to locate the “hot spot” in the brain where the seizures have been originating. They do so with the help of electrodes placed in the brain and listen for telltale electrical storms. Locating the “hot spot” can take several weeks. In the meantime, electrodes are implanted in or near the areas of the brain that are involved in the movement as well as auditory signaling.

The five patients recruited for the experiment agreed to undergo trial for the new voice system. Each of the participants was implanted with one or two electrodes placed on the surface of the brain.

Each participant recited hundreds of sentences and the electrodes started recording the firing patterns of the neurons in the brain. The scientists associated those patterns with the subtle movements of their lips, tongue, larynx, and jaw, which occur during a natural speech. They then translated the subtle movements into spoken sentences. The team is now planning to conduct extensive clinical trials. However, the biggest challenge could be finding suitable patients, such as those who have strokes and a speech disability.