A community-developed open-source computational ecosystem for big neuro data


Abstract

Recent technological developments, such as high-throughput imaging and sequencing, enable experimentalists to collect increasingly large, complex, and heterogeneous ‘big’ data. These studies result in terabytes of data per day, yielding petabytes across experiments and laboratories. These experimental capabilities exceed the scale or feature set of existing software. For example, such data cannot be stored, processed, and visualized on a laptop or workstation. Instead, big data must be stored on data centers and processed on high-performance compute clusters.In 2011, we launched Open Connectome Project1, an open-access data repository powered by open-source web-services software applications that store, analyze, and visualize large imaging datasets. However, as technology changed, features were added, and scale increased, our academic development team and resources became overwhelmed. We overhauled our custom stack into a community-built and -maintained software ecosystem deployed in the commercial cloud, integrating multiple open-source projects and extending them for our needs (https://neurodata.io). The ecosystem makes it possible to analyze disparate datasets by reusing components originally designed for other applications.

Citation

@articleVogelstein_2018 doi: 10.1038/s41592-018-0181-1 url: https://doi.org/10.1038/s41592-018-0181-1 year: 2018 month: oct publisher: Springer Science and Business Media LLC volume: 15 number: 11 pages: 846--847 author: Vogelstein Joshua T. and Perlman Eric and Falk Benjamin and Baden Alex and Roncal William Gray and Chandrashekhar Vikram and Collman Forrest and Seshamani Sharmishtaa and Patsolic Jesse L. and Lillaney Kunal and Kazhdan Michael and Hider Robert and Pryor Derek and Matelsky Jordan and Gion Timothy and Manavalan Priya and Wester Brock and Chevillet Mark and Trautman Eric T. and Khairy Khaled and Bridgeford Eric and Kleissas Dean M. and Tward Daniel J. and Crow Ailey K. and Hsueh Brian and Wright Matthew A. and Miller Michael I. and Smith Stephen J. and Vogelstein R. Jacob and Deisseroth Karl and Burns Randal title: A community-developed open-source computational ecosystem for big neuro data journal: Nature Methods

Contact Us


Chief
Ashley Llorens
Ashley.Llorens@jhuapl.edu
240-228-0312

Physical Address
7701 Montpelier Road
Laurel, MD 20723


The Intelligent Systems Center is located at the Montpelier Campus of the Johns Hopkins Applied Physics Laboratory.
Click here for a map, directions and other visitor information.