phd thesis
11 Signposts5 Saplings5 Chests4 Seedlings2 Trees1 Withered1 StoneCharacterizing Unstructured Motor Behaviors in the Epilepsy Monitoring Unit, by Dr. Paolo Gutierrez Gabriel, published 2019. It is not longer under embargo.
How I completed my dissertation
For my graduate research, I investigated the generalization of brain-machine interfaces in less constrained environments. At the time, these interfaces required tight, limiting conditions for participants to use (e.g. sitting extremely still while hands are on the table).
Summary
This work leveraged my engineering and research toolkits to acquire multimodal datasets (RGB-D, iEEG), extract meaningful behavioral labels, and decode said behaviors using high-fidelity neural signals.
See the following project pages for more details:
- Continuous Behavior Recording
- Tracking Unconstrained Movement Kinematics in the Epilepsy Monitoring Unit
- Neural Correlates to Unconstrained Movement in the Epilepsy Monitoring Unit
- 3D Brain Mapping
Continuous Behavior Recording
Clinical recording system. Continuous RGB-D video of epilepsy patient in EMU. Micro-controller ensures time alignment of video to clinical neural recordings.
OBJECTIVE : Study the human brain in a naturalistic setting by recording RGB-D video alongside brain signals from electrodes implanted in the brain of human (epileptic) subjects.
SCOPE:
- Identified technical constrains of long-term continuous recording
- Designed recording system using commercially available components
- Deployed recording system in multiple (2) epilepsy units
- Demonstrate the systems ability to collect research-relevant data:
- Tracked unconstrained patient movements
- Neural correlates to unconstrained movements
Tracking Unconstrained Movement Kinematics in the Epilepsy Monitoring Unit
Computer vision algorithms to semi-automatically track natural movement. Dense optical flow, Caffe-Heatmap, etc.
OBJECTIVE : Provided detailed kinematic accounts of unconstrained movement. Do so in a semi-automatic way to avoid annotating days of recorded video.
SCOPE:
- Coarse L/R movement tracking using KLT feature tracker
- Movement onset detection using Dense Optical Flow (Gunner-Farneback, DeepFlow)
- Upper-body pose estimation using Convolutional Neural Networks (Caffe-Heatmap)
- Object tracking for context labeling
Neural Correlates to Unconstrained Movement in the Epilepsy Monitoring Unit
Exploring the relationship between neural activity and unconstrained movements. The brain is recorded from various regions differeing across subjects.
OBJECTIVE : Identify neural activity that modulates with respect to unconstrained human movement. Compare these features to those previously published using constrained experimental tasks.
SCOPE:
- Decoding movement onset using common task-based features
- Neural response to changes in kinematics
- Passive brain mapping of unconstrained behavior
- Moment-by-moment decoding of unconstrained movements
3D Brain Mapping
I use iELVis and FreeSurfer to merge pre/post-operative head scans to model implanted brains.
We needed to know from where in the brain our signals were coming. To do this, Eric Halgren's lab lent their expertise with 3D modeling of brain scans and localizing electrode placement.
Here is an example of one of the subjects, the electrodes were placed at different depths in the frontal region of the left hemisphere.
The cool thing about making 3D brain models from raw MRI data is that you can print them. Here is a print of my own brain, courtesy of a friend I met through the National Center for Adaptive Neurotechnologies!
Published work
- Gabriel, P., Doyle, W.K., Devinsky, O., Friedman, D., Thesen, T. and Gilja, V. (2016) ”Neural correlates to automatic behavior estimations from RGB-D video in epilepsy unit.” In Engineering in Medicine and Biology Society (EMBC), 2016 IEEE 38th Annual International Conference (pp. 3402-3405). IEEE.
- Chen, K.J., Gabriel, P., Alasfour, A., Gong, C., Doyle, W.K., Devinsky, O., Friedman, D., Dugan, P., Melloni, L., Thesen, T., Gonda, D., Sattar, S., Wang, S.G., and Gilja, V. (2018) ”Patient-Specific Pose Estimation in Clinical Environments.” IEEE Journal of Translational Engineering in Health and Medicine. IEEE.
- Alasfour, A., Gabriel, P., Jiang, X., Shamie, I., Melloni, L., Thesen, T., Dugan, P., Friedman, D., Doyle, W., Devinsky, O., Gonda, D., Sattar, S., Wang, S., Halgren, E., and Gilja, V. (2019) "Coarse behavioral context decoding." Journal of Neural Engineering (vol 16.1, 016021).
- Gabriel, P., Chen, K., Alasfour, A., Pailla, T., Doyle, W., Devinsky, O., Friedman, D., Dugan, P., Melloni, L., Thesen, T., Gonda, D., Sattar, S., Wang, S.G, and Gilja, V. (2019) "Neural correlates of unstructured motor behaviors." Journal of Neural Engineering (vol 16.6, 066026).
- Alasfour, A., Gabriel, P., Jiang, X., Shamie, I., Melloni, L., Thesen, T., Dugan, P., Friedman, D., Doyle, W., Devinsky, O. and Gonda, D. (2022) "Spatiotemporal dynamics of human high gamma discriminate naturalistic behavioral states." PLoS Computational Biology (vol 18.8, p.e1010401).
- Martin, J., Gabriel, P., Gold, J., Haas, R., Davis, S., Gonda, D., Sharpe, C., Wilson, S., Nierenberg, N., Scheuer, M. and Wang, S. (2022) "Optical flow estimation improves automated seizure detection in neonatal EEG." Journal of Clinical Neurophysiology (vol 39.3, pp.235-239).