Launching ndstore in EC2

ndstore is a spatial database tool which can be deployed in the cloud and allows users to store, query, and download massive image volumes easily and quickly. Previously inaccessible images, such as sections off of a serial electron microscope, can thus be easily manipulated for processing and visualization. Prior to beginning this tutorial, it is recommended that you are sure that launching a database is what you want to do. This comes with management, and the cost associated with running the service indefinitely. In the case of most users, you wish to simply store and access your data in the cloud. If that better describes your usecase, please checkout our autoingest tutorial, instead.

I'll say it again in just to be sure the message gets across:

This is not the tutorial you want to follow if simply you wish to store your data. This is launching your own database which requires management and the costs associated with persistent cloud instances. Proceed at your own risk

Step 1: Launch an instance

In order to standup ndstore on EC2 we must first launch a compatible instance of sufficient size. In this example, we have chosen an Ubuntu 14.04 (Trusty) image (ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-20161130 (ami-1081b807)), the instance size t2.large, and the root ssd size set to 15 GB. We have done nothing to modify the defaults for security groups or any other features of the instance.

Once you launch your instance, you can connect to it like any other EC2 instance, via SSH, with a command similar to that shown below:

ssh -i /path/to/your/key.pem ubuntu@123.45.678.90

In the above you must substitute /path/to/your/key.pem with the location of the private key you registered to the instance, and 123.45.678.90 with the ip address of your instance. Both of these details can be found within Amazon's EC2 instance console when the current instance is selected.

Step 2: Install ndstore

Once you connect to the instance, you should update packages on the system and then begin installing ndstore. Copy-pasting the following commands exactly as-is will do this for you.

sudo apt-get update -y   # updates packages
sudo apt-get install -y git   # installs git client
git clone https://github.com/neurodata/ndstore.git ndstore   # clones ndstore git repository
cd ndstore   # navigates into repository
git checkout microns   # changes to the relevant branch of ndstore
cd setup   # navigates into setup directory
sudo ./ndstore_install.sh   # begins install

You may see that some of the tests fail at the end of the installation. The primary reason for this currently is that the tests are being run against a development server despite being designed for nginx, so HTTP and HTTPS protocol inconsistencies result in these failures. Of course, telling you that tests may fail doesn't indicate failure isn't particularly helpful, but that's why he have the next step to test...

Step 3: Use the database

Now that your database has been installed and configured, you should head over to the s3ingest tutorial to begin putting data in your service.