In [ ]:
In [4]:
%reset -f
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import pandas as pd
%matplotlib inline
import seaborn as sns
sns.set(style="whitegrid", palette="muted", color_codes=True)
import tensorflow as tf;
print('TF version:' + tf.__version__)
In [3]:
import os
print("Path at terminal when executing this file")
print(os.getcwd() + "\n")
%matplotlib inline
%load_ext watermark
%watermark -d -v -m -p pandas,scipy,matplotlib
apt-get install docker.io
https://download.docker.com/mac/stable/Docker.dmg
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install caskroom/cask/brew-cask
brew cask install docker-toolbox
In [ ]:
Base images are images that have no parent image, usually images with an OS like ubuntu, busybox or debian.
We start with specifying our base image. Use the FROM keyword to do that
FROM ubuntu:16.04
docker pull ubuntu:12.04
In [ ]:
RUN pip --no-cache-dir install \
http://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.0.0-cp27-none-linux_x86_64.whl
In [ ]:
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
curl \
libfreetype6-dev \
libpng12-dev \
libzmq3-dev \
pkg-config \
python \
python-dev \
rsync \
software-properties-common \
unzip \
&& \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
In [ ]:
RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
python get-pip.py && \
rm get-pip.py
In [ ]:
RUN pip --no-cache-dir install \
ipykernel \
jupyter \
matplotlib \
numpy \
scipy \
sklearn \
pandas \
Pillow \
&& \
python -m ipykernel.kernelspec
In [ ]:
COPY jupyter_notebook_config.py /root/.jupyter/
In [ ]:
# TensorBoard
EXPOSE 6006
# IPython
EXPOSE 8888
In [ ]:
# jupyter_notebook_config.py
import os
from IPython.lib import passwd
c.NotebookApp.ip = '*'
c.NotebookApp.port = int(os.getenv('PORT', 8888))
c.NotebookApp.open_browser = False
# sets a password if PASSWORD is set in the environment
if 'PASSWORD' in os.environ:
c.NotebookApp.password = passwd(os.environ['PASSWORD'])
del os.environ['PASSWORD']
Docker consists of the following components:
Images
Containers
Daemon
Clients
Registries
Images are read-only templates which provide functionality for running an instance of this image (container). An example for a image is the latest release of Ubuntu. Images are defined as layers, for example, you can add Java to the Ubuntu image and get another image based on this.
The Docker hub provides lot of pre-configured images. You can modify existing images and save these modifications as new image.
Container are the started components based on images. They contain the actual application and dependencies but share the same kernel. They can be started, stopped, paused, deleted. Containers are immutable and disposable.
Is used to manage the container. It runs natively on Linux and inside a VM on Windows and Mac OS X. To start it use the docker command.
Clients (CLI, IDE) run on host VM. They provide the tools to interact with container, i.e., to start them.
Images are saved in a registry and have an ID with consists of a repository and a tag. For example, fedora:22, is an image which contains the Fedora 22 OS from the fedora repository.
To use an image you have to pull it from a registry, to share an image with others you have to push it to one. The default Docker registry is the Docker Hub. You can upload your personal images to Github, in this case you add your user name as prefix to the image, e.g., vogella/fedore:22
In [ ]:
root@shlomo::~# docker search --stars=5 "postgresql-9.3"
Flag --stars has been deprecated, use --filter=stars=3 instead
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
helmi03/docker-postgis PostGIS 2.1 in PostgreSQL 9.3 24 [OK]
In [ ]:
docker build -t quantscientist/deep-ml-meetups -f Dockerfile.cpu .
In [ ]:
root@shlomo:~# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
quantscientist/deep-ml-meetups latest 3871333c6375 5 weeks ago 5.146 GB
In [ ]:
root@shlomo:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker-airflow latest 8117652bf8e8 7 weeks ago 1.22 GB
debian jessie 8cedef9d7368 2 months ago 123 MB
tylerfowler/superset latest 224639f0ff97 4 months ago 879 MB
uifd/ui-for-docker latest 965940f98fa5 8 months ago 8.1 MB
cloudera/clusterdock latest 3e15a0e12577 8 months ago 463 MB
zhicwu/presto latest a1fe6f0241a4 12 months ago 2.05 GB
cloudera/quickstart latest 4239cd2958c6 13 months ago 6.34 GB
In [ ]:
#!/bin/bash
# Delete all containers
docker rm $(docker ps -a -q)
# Delete all images
docker rmi $(docker images -q)
In [ ]:
# Delete all stopped containers
docker rm $( docker ps -q -f status=exited)
# Delete all dangling (unused) images
docker rmi $( docker images -q -f dangling=true)
In [ ]:
docker exec -it <containerIdOrName> bash
( Docker version 1.3 or greater)
In [ ]:
docker network ls
In [ ]:
FROM nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04
In [ ]:
shlomokashani@tf-docker-meetup:~/dev/deep-ml-meetups/nice-docker$ history
1 mkdir dev
2 cd dev/
3 apt-get install docker.io
4 sudo apt-get install docker.io
5 git clone git@github.com:QuantScientist/deep-ml-meetups.git
6 ssh-keygen -t rsa -b 4096 -C "shlomokashani@gmail.com"
7 cat /home/shlomokashani/.ssh/id_rsa.pub
8 ssh-keygen -t rsa -b 4096 -C "shlomokashani@gmail.com"
9 git clone git@github.com:QuantScientist/deep-ml-meetups.git
10 cd deep-ml-meetups/nice-docker/
11 docker build -t quantscientist/deep-ml-meetups -f Dockerfile.cpu .
12 sudo docker build -t quantscientist/deep-ml-meetups -f Dockerfile.cpu .
13 docker ps
14 sudo usermod -aG docker shlomokashani
15 history
In [ ]:
docker run -it -p 5555:5555 -p 7842:7842 -p 8787:8787 -p 8786:8786 -p 8788:8788 -v ~/dev/:/root/sharedfolder quantscientist/deep-ml-meetups bash
In [ ]:
shlomokashani@cloudshell:~$ gcloud config set project tf-docker
Updated property [core/project].
shlomokashani@tf-docker:~$ datalab create tf-docker-datalab
In [ ]:
git clone https://github.com/GoogleCloudPlatform/datalab.git
cd datalab/containers/datalab
# Replace the MyProjectID value in the next line with your project ID
PROJECT_ID=MyProjectID
./build.sh && ./run.sh