Shared Neural Manifolds Improve Speech Decoding Across Individuals

Introduction: Recent developments in neural recording technologies have revolutionized language neurobiology and motor speech enabling brain-computer interfaces (BCI) that restore speech functions. Real-time speech decoding models, typically customized to individual data from intact sensorimotor cortex, face challenges in generalizing, particularly for aphasia patients whose language networks are disrupted by lesions.

Methods: To address this, we developed group sequence-to-sequence (seq2seq) models trained on datasets from individuals with normal language function, using stereo-electroencephalography to decode activity from distributed speech hubs. This approach significantly reduced phoneme error rate (PER) for all subject (n=7) on held-out trials when initialized with the group-level manifold. Building on these insights, we explored these models' clinical applicability for aphasia with transfer learning techniques to adapt shared latent neural dynamics from a group model of patients with sensorimotor cortex coverage to a single-subject model with differential coverage.

Results: Despite the incomplete sampling of necessary cortical areas involved in encoding articulatory kinematics, this adaptation decreased PER from 61% to 36% showcasing the potential of using pre-learned neural states for phoneme mappings to enhance single-subject decoding performance. This use of shared latent mappings presents a robust approach to adapt and apply aggregated neural encodings to situations where there is missing or incomplete sampling of cortical dynamics, extending BCI utility to those with significant cortical impairments. We used group-level dynamics to simulate artificial electrode densities across critical regions such as the posterior superior temporal, subcentral, and inferior frontal cortices, to infer optimal design of neural implants for assistive communication. Furthermore, our approach supports a causality analysis through region and lesion-based assessments to determine the necessity and sufficiency of specific cortical areas.

Conclusion : This comprehensive methodology not only advances BCI design and neural decoding techniques but also has profound implications for personalized medicine and rehabilitative strategies, significantly enhancing the quality of life and communication capabilities for individuals with speech impairments.