Feature Selection Methods for Zero-Shot Learning of Neural Activity
Figure 1. Electrode placement grid for the ECoG participants overlaid on a magnetic resonance imaging (MRI) scan for anatomical reference. Each electrode leads to a time varying activation signal from which features are extracted.
Dimensionality poses a serious challenge when making predictions from human neuroimaging data. Across imaging modalities, large pools of potential neural features (e.g., responses from particular voxels, electrodes, and temporal windows) have to be related to typically limited sets of stimuli and samples (Figure 1).
In recent years, zero-shot prediction models have been introduced for mapping between neural signals and semantic attributes, which allows for classification of stimulus classes not explicitly included in the training set. While choices about feature selection can have a substantial impact when closed-set accuracy, open-set robustness, and runtime are competing design objectives, no systematic study of feature selection for these models has been reported. Instead, a relatively straightforward feature stability approach has been adopted and successfully applied across models and imaging modalities. To characterize the trade-offs in feature selection for zero-shot learning, we compared correlation-based stability to several other feature selection techniques on comparable data sets from two distinct imaging modalities: functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG). While most of the feature selection methods resulted in similar zero-shot prediction accuracies and spatial/spectral patterns of selected features, there was one exception; a novel feature/attribute correlation approach was able to achieve those accuracies with far fewer features, suggesting the potential for simpler prediction models that yield high zero-shot classification accuracy (Figure 2).
Figure 2. Peak MRA achieved for encoding (top) and decoding (bottom). Each group of bars corresponds to a different participant, and the vertical axis represents MRA. Each colored bar represents a different feature selection method.
Further analysis seems to indicate that this improvement might be the result of a more diverse spatial distribution of the chosen features (in fMRI, see Figure 3) or a more diverse temporal distribution (in ECoG).
Figure 3. Voxel selection by method for the first fMRI participant. Each subpanel shows the three-dimensional spatial scatter of voxel positions, with each voxel colored proportionally to the number of LOCO cross-validation folds it was selected.
For more information on the experimental results, please refer to reference 2.
This MATLAB code may be used to replicate the results published in the manuscript (reference 1):
The code is organized into three directories as follows:
code/ – code for replicating the experiments from the paper
functions/ – helper functions used to perform feature selection, model learning, and path management
scripts/ – scripts to run the various experiments and summarize results
third_party/ – external third party toolboxes used by our code
data/ – neural data used to run the experiments
fMRI/ – fMRI data used in Mitchell (2008), reformatted to work with our code
ECoG/ – ECoG data collected at the Johns Hopkins University
To run the code:
Execute the following scripts:
To generate additional figures, run the summary scripts:
The scripts will generate figures and outputs, and create the following directories to save them:
output/Results – figures summarizing results
output/Tmp – binaries saving out the MATLAB workspace
Zero-Shot is available for download.
Please fill out the required information on the form below to receive an e-mailed copy of the zip file. Your submission of this form will serve as the acceptance of the End User License Agreement (EULA). It is intended that this code be freely available for academic, personal education, and research purposes. Any other uses, especially for commercial purposes, should be discussed with the APL Office of Technology Transfer. We also ask that you follow this process to receive the code rather than sharing copies so that we can keep track of usage.
 C. A. Caceres, M. J. Roos, K. M. Rupp, G. Milsap, N. E. Crone, M. E. Wolmetz, and C.R. Ratto, “Feature Selection Methods for Zero-Shot Learning of Neural Activity,” Frontiers in Neuroinformatics, 2017
 Kyle Rupp, Matthew Roos, Griffin Milsap, Carlos Caceres, Christopher Ratto, Mark Chevillet, Nathan E. Crone, Michael Wolmetz, “Semantic Attributes Are Encoded in Human Electrocorticographic Signals during Visual Object Recognition,” NeuroImage, Volume 148, 1 March 2017, pages 318–329, ISSN 1053-8119, https://doi.org/10.1016/j.neuroimage.2016.12.074.