Ndstore Save

Scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data called Neurodata Web Services(formerly called the Open Connectome Project).

Major directories include:

• spdb -- Spatial Database submodule
• ndingest -- AWS Wrappers for Parallel Ingest submodule
• ndlib -- Common library and Ctype accelerations submodule
• ingest-client -- Ingest client and related services submodule
• webservices -- Webservices module
• ndproj -- Project Module
• ramon -- RAMON and Annotation metadata module
• django -- Django module
• setup -- Install script, setup instructions and configurations
• test -- Test module
• examples -- How to use the service
• admin -- Scripts to manage the databases
• docs -- Sphinx documentation for the project
• scripts -- Useful general purpose scripts

NOTE: git submodule init and git submodule update for all submodules

Architecture

Installation

• Use a fresh Ubuntu14.04 instance with sudo access.

wget https://raw.githubusercontent.com/neurodata/ndstore/master/setup/ndstore_install.sh
sudo chmod 7000 ndstore_install.sh
sudo ./ndstore_install.sh

Note: ndstore does not support Docker images because running custom systemd services needs some relaxation of security on the Docker engine leading to issues. We recommend using the install script used above

Benchmarks

The benchmarks were performed on AWS EC2 instance type i2.8xlarge with a MySQL backend.

Read Throughput

Write Throughput