Synthesis of imagined speech processes from minimally invasive recordings of neural activity
2021
Maastricht University, Maastricht, Netherlands(1)
University of Bremen, Bremen, Germany(2)
University of Bremen, Bremen, Germany(2)
Speech neuroprosthetics aim to provide a natural communication channel to individuals who are unable to speak due to physical or neurological impairments. Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and notably improve quality of life, particularly for individuals who have severely limited means of communication. Recent advances in decoding approaches have led to high-quality reconstructions of acoustic speech from invasively measured neural activity. However, most prior research utilizes data collected during open-loop experiments of articulated speech, which might not directly translate to imagined speech processes. Here, an approach that synthesizes audible speech in real-time for both imagined and whispered speech conditions was presented. Using a participant implanted with stereotactic depth electrodes, the authors were able to reliably generate audible speech in real-time. The decoding models rely predominately on frontal activity suggesting that speech processes have similar representations when vocalized, whispered, or imagined. While reconstructed audio is not yet intelligible, the real-time synthesis approach represents an essential step toward investigating how patients will learn to operate a closed-loop speech neuroprosthesis based on imagined speech.
Real-time synthesis of imagined speech processes from minimally invasive recordings of neural activity
Christian Herff(1), Miguel Angrick(2)
Added on: 03-01-2022
[1] https://www.nature.com/articles/s42003-021-02578-0
