Brain waves: a neural suit with a big vocabulary

Neuroprosthetics translate nerve signals directly from the brain, for example to control hand prostheses. The power of thought can also be used to move a cursor across a screen or to maneuver a wheelchair through space. In the future, interfaces should help paralyzed people to communicate directly with their environment again. Because in many sufferers, the ability to speak is completely intact, for example in as such– o Imprisoned-Patients.

For years, extensive research has been done on interfaces that are supposed to “understand” imaginary language. Among other things, it was already possible to have simple conversations between them At least two people to translate in many cases. Such experiments are usually done with epilepsy patients who have electrodes in their brain due to their disease. The activity of some neurons can be read directly there. Even non-invasive methods, such as electroencephalography (electroencephalogram), is now being tested.

Electrodes in the brain

Other studies work directly with those affected, such as the one just published in the journal Nature Communications. work from the team Edward Chang From the University of California, San Francisco: The subject was a 36-year-old patient who was almost completely paralyzed after suffering a severe stroke and could no longer speak. With the help of only small head movements, it operates a computer-aided interface.

For the experiments, a credit card-sized implant containing 128 electrodes was inserted into the young man without cognitive problems. The researchers already achieved their first success last year In the New England Journal of Medicine mentioned. The program used at that time was developed into nearly 50 training units with the help of “deep learning” It was trained while the patient was trying to pronounce one of the 50 given words.

In addition to machine learning methods from artificial intelligence, classical natural language models have also been used, which, for example, indicate the probability of words occurring in a given context. In this way, the program has already learned to convert brainwaves into speech. In the end, it was possible to get acquainted with whole sentences of limited vocabulary. However, the error rate was a quarter.

to speak mentally

The approach tested for the current study should not only be more precise and comprehensive, but also more direct. This means that the test person no longer had to try mentally to actually speak – specifically also physically – but was able to think of the word or letter directly. This makes the process less unnatural, and therefore possibly more suitable for the daily use of those affected, write Chang and Co.

Incidentally, other research groups are also trying to make the brain more readable by taking a turn through the motor system, for example in the form of “mental writing” – like pure handwriting, like Team reported last year. As new work now suggests, it may not matter to a computer whether the brain produces words mechanically or mentally.

Few letters, many words

Chang’s team also found a fairly simple trick to increase the vocabulary of neurocompensation without much effort. Instead of thinking of the full words, this time the subject had to spell them out. The study authors assure that this would allow a much larger number of words to be represented with just 26 letters. In order for the machine to “understand” the characters better, the International Alphabet Table (“Alpha, Bravo, Charlie, …”) was used. For training and tests, artificial intelligence and classical models of language processing were combined again.

Indeed, in the vast majority of cases, the computer was able to recognize mental sentences generated by the method of letters from a basic vocabulary of 1,152 words. The average error rate per character was just over six percent, and nearly 30 characters could be processed per minute. This vocabulary will be very useful for everyday communication. Simulation calculations made by the researchers show that vocabulary can be increased to 9,000 words without heavy losses. Until then, the average error rate will be just over eight percent.