Decoding Hand Movement Patterns from M1 Electrocorticographic Signals Using Deep Learning

Introduction: Uncovering the complex relationship between neural signals and behavior is essential to elucidating motor cortex organization and advancing brain-computer-interfaces. Our study examines the application of deep learning algorithms for decoding discrete hand gestures from real-time M1 electrocorticographic activity.

Methods: A patient undergoing DBS lead placement had a 1024-contact cortical surface array temporarily placed over the hand-knob in the primary motor cortex (M1). They performed rock, paper, and scissor hand gestures while wearing a motion capture glove. Cortical activity and hand position were recorded and synchronized using custom software. Convolutional neural networks (CNNs) were developed to identify (i) movement versus non-movement, (ii) hand gestures, and (iii) gesture transitions. For (i), a three-layer CNN with 3x3 kernels and ReLU activations was trained on gamma bands alone, followed by 2-4 temporal and spatial layers trained on seven power spectra feature bands spanning 9-130 Hz. For (ii), a three-layer CNN with 5x5 kernels and ReLU activations was trained on gamma and beta bands spanning 12-80 Hz. For (iii), a CNN processed frequency bands spanning 2-150 Hz within 400-600 msec of movement initiation, independently across four parallel layers, each configured to a specific frequency band.

Results: Performance in the CNN model using gamma alone to differentiate between movement versus non-movement consistently achieved accuracy >75%. Incorporating temporal features in sliding detection, performance increased to >80%. Using the same data, distinct CNN models were trained to predict specific hand gestures with an accuracy of 47% and gesture transitions >85%.

Conclusion : Our findings demonstrate that CNNs can be used to effectively decode hand gestures from surface electrocorticography; however, there may be room for improvement through identifying the behaviorally relevant and dynamic neural signals. Understanding consistent features across individuals will be important to allow future models trained on able-bodied patients to be deployed on patients with neurologic injury such as spinal cord injury.