How can a computer translate thoughts into spoken words? A team of scientists from the University of California have discovered a new and promising piece of that puzzle, and the result is a surprisingly convincing pattern of synthetic speech.
The scientists created a system that translates brain waves into words by focusing on the physical movements related to speech, rather than the sound of the words trying to be communicated. They found looking at the intended movements of the tongue, larynx and other speech mechanisms allowed them to reproduce voice sounds in a more reliable way than, say, trying to match brain waves to predicted speech sounds.
Using this information, the team created a computer program that simulates the movements of a vocal tract by honing in on the brain’s speech centers.
Take a look at an example of this type of speech modeling. You can see the connection between the intended spoken words, and the way those words are formed by the different parts of the vocal tract.
The team’s findings were recently published in the journal Nature. The paper concluded that this new method could be the basis of life-changing technology for people with severe speech disorders, physical trauma or other conditions that limit their ability to communicate.
“It’s been a longstanding goal of our lab to create technologies to restore communication for patients with severe speech disability,” Edward Chang, one of the project’s co-authors, said in a press briefing. “We want to create technologies that can reproduce speech directly from human brain activity. This study provides a proof of principle that this is possible.”
That’s not the only exciting takeaway from the team’s research. According to Chang, their model of the mechanical speech process could actually be applied from one person to another.
“The neural code for vocal movements is partially shared across different individuals, and that an artificial vocal tract modeled on one person’s voice can be adapted to synthesize the speech from another person’s brain activity,” Chang explains. “This means that a speech decoder that’s trained in one person with intact speech could maybe someday act as a starting point for someone who has speech disability who could then learn to control the simulated vocal tract using their own brain activity.”
According to another recent study cited by the UC scientists, communication technologies for people with speech and motor limitations are evolving, but can still be frustrating and inaccurate. If this latest breakthrough can eventually be applied on an individual patient level, it could open up a new world of understanding, and being understood.
