Feature Selection Methods for Zero-Shot Learning of Neural Activity
Dimensionality poses a serious challenge when making predictions from human neuroimaging data. Across imaging modalities, large pools of potential neural features (e.g. responses from particular voxels, electrodes, and temporal windows) have to be related to typically limited sets of stimuli and samples [Fig 1].
Figure 1: Electrode placement grid for the ECoG participants overlaid on a magnetic resonance imaging (MRI) scan for anatomical reference. Each electrode leads to a time varying activation signal from which features are extracted.
In recent years, zero-shot prediction models have been introduced for mapping between neural signals and semantic attributes, which allows for classification of stimulus classes not explicitly included in the training set. While choices about feature selection can have a substantial impact when closed-set accuracy, open-set robustness, and runtime are competing design objectives, no systematic study of feature selection for these models has been reported. Instead, a relatively straightforward feature stability approach has been adopted and successfully applied across models and imaging modalities. To characterize the tradeoffs in feature selection for zero-shot learning, we compared correlation-based stability to several other feature selection techniques on comparable data sets from two distinct imaging modalities: functional Magnetic Resonance Imaging (fMRI) and Electrocorticography (ECoG). While most of the feature selection methods resulted in similar zero-shot prediction accuracies and spatial/spectral patterns of selected features, there was one exception; A novel feature/attribute correlation approach was able to achieve those accuracies with far fewer features, suggesting the potential for simpler prediction models that yield high zero-shot classification accuracy [Fig 2].
Figure 2: Peak MRA achieved for encoding (top) and decoding (bottom). Each group of bars corresponds to a different participant, and the vertical axis represents MRA. Each colored bar represents a different feature selection method.
Further analysis seems to indicate that this improvement might be the result of a more diverse spatial distribution of the chosen features (in fMRI, see [Fig 3]) or a more diverse temporal distribution (in ECoG).
Figure 3: Voxel selection by method for the first fMRI participant. Each sub-panel shows the three-dimensional spatial scatter of voxel positions, with each voxel colored proportionally to the number of LOCO cross-validation folds it was selected.
For more information on the experimental results please refer to reference 2.
This MATLAB code may be used to replicate the results published in the manuscript [Ref 1]:
The code is organized into three directories as follows:
- code/ - Code for replicating the experiments from the paper.
- functions/ - helper functions used to perform feature selection, model learning, and path management
- scripts/ - scripts to run the various experiments and summarize results
- third_party/ - external third party toolboxes used by our code
- data/ - Neural data used to run the experiments
- fMRI/ - fMRI data used in Mitchell (2008), reformatted to work with our code
- ECoG/ - ECoG data collected at JHU
To run the code:
- Execute the following scripts:
- To generate additional figures, run the summary scripts:
- The scripts will generate figures and outputs, and create the following directories to save them:
- output/Results - figures summarizing results
- output/Tmp - binaries saving out the MATLAB workspace
Zero-Shot is available for download.
Please fill out the required information on the form below to receive an emailed copy of the zip file. Your submission of this form will serve as the acceptance of the End User License Agreement (EULA). It is intended that this code be freely available for academic, personal education, and research purposes. Any other uses, especially for commercial purposes, should be discussed with JHU/APL Office of Technology Transfer. We also ask that you follow this process to receive the code rather than sharing copies so that we can keep track of usage.
-  C.A. Caceres, M.J. Roos, K.M. Rupp, G. Milsap, N.E. Crone, M.E. Wolmetz, and C.R. Ratto, "Feature Selection Methods for Zero-Shot Learning of Neural Activity," Frontiers in Neuroinformatics, 2017
-  Kyle Rupp, Matthew Roos, Griffin Milsap, Carlos Caceres, Christopher Ratto, Mark Chevillet, Nathan E. Crone, Michael Wolmetz, Semantic attributes are encoded in human electrocorticographic signals during visual object recognition, NeuroImage, Volume 148, 1 March 2017, Pages 318-329, ISSN 1053-8119, https://doi.org/10.1016/j.neuroimage.2016.12.074.