Setup and customize deep learning environment in seconds.
Deepo2 is now a series of Docker images that
and their Dockerfile generator that
docker pull ufoym/deepo
Now you can try this command:
nvidia-docker run --rm ufoym/deepo nvidia-smi
This should work and enables Deepo to use the GPU from inside a docker container. If this does not work, search the issues section on the nvidia-docker GitHub -- many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do
nvidia-docker run -it ufoym/deepo bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
nvidia-docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash
This will make /host/data
from the host visible as /data
in the container, and /host/config
as /config
. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with --ipc=host
or --shm-size
command line options to nvidia-docker run
.
nvidia-docker run -it --ipc=host ufoym/deepo bash
You are now ready to begin your journey.
$ python
>>> import tensorflow
>>> import sonnet
>>> import torch
>>> import keras
>>> import mxnet
>>> import cntk
>>> import chainer
>>> import theano
>>> import lasagne
>>> import caffe
$ caffe --version
caffe version 1.0.0
$ th
│ ______ __ | Torch7
│ /_ __/__ ________/ / | Scientific computing for Lua.
│ / / / _ \/ __/ __/ _ \ | Type ? for help
│ /_/ \___/_/ \__/_//_/ | https://github.com/torch
│ | http://torch.ch
│
│th>
Note that docker pull ufoym/deepo
mentioned in Quick Start will give you a standard image containing all available deep learning frameworks. You can customize your own environment as well.
If you prefer a specific framework rather than an all-in-one image, just append a tag with the name of the framework. Take tensorflow for example:
docker pull ufoym/deepo:tensorflow
Note that all python-related images use Python 3.6
by default. If you are unhappy with Python 3.6
, you can also specify other python versions:
docker pull ufoym/deepo:py27
docker pull ufoym/deepo:tensorflow-py27
Currently, we support Python 2.7
and Python 3.6
.
See https://hub.docker.com/r/ufoym/deepo/tags/ for a complete list of all available tags. These pre-built images are all built from docker/Dockerfile.*
and circle.yml
. See How to generate docker/Dockerfile.*
and circle.yml
if you are interested in how these files are generated.
git clone https://github.com/ufoym/deepo.git
cd deepo/generator
pip install -r requirements.txt
For example, if you like pytorch
and lasagne
, then
python generate.py Dockerfile pytorch lasagne
This should generate a Dockerfile that contains everything for building pytorch
and lasagne
. Note that the generator can handle automatic dependency processing and topologically sort the lists. So you don't need to worry about missing dependencies and the list order.
You can also specify the version of Python:
python generate.py Dockerfile pytorch lasagne python==3.6
docker build -t my/deepo .
This may take several minutes as it compiles a few libraries from scratch.
. | modern-deep-learning | dl-docker | jupyter-deeplearning | Deepo |
---|---|---|---|---|
ubuntu | 16.04 | 14.04 | 14.04 | 16.04 |
cuda | :x: | 8.0 | 6.5-8.0 | 8.0 |
cudnn | :x: | v5 | v2-5 | v6 |
theano | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
tensorflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
sonnet | :x: | :x: | :x: | :heavy_check_mark: |
pytorch | :x: | :x: | :x: | :heavy_check_mark: |
keras | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
lasagne | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
mxnet | :x: | :x: | :x: | :heavy_check_mark: |
cntk | :x: | :x: | :x: | :heavy_check_mark: |
chainer | :x: | :x: | :x: | :heavy_check_mark: |
caffe | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
torch | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
Deepo is a Docker image with a full reproducible deep learning research environment. It contains most popular deep learning frameworks: theano, tensorflow, sonnet, pytorch, keras, lasagne, mxnet, cntk, chainer, caffe, torch.
You can either directly download the image from Docker Hub, or build the image yourself.
docker pull ufoym/deepo
git clone https://github.com/ufoym/deepo.git
cd deepo && docker build -t ufoym/deepo .
Note that this may take several hours as it compiles a few libraries from scratch.
Now you can try this command:
nvidia-docker run --rm ufoym/deepo nvidia-smi
This should work and enables Deepo to use the GPU from inside a docker container. If this does not work, search the issues section on the nvidia-docker GitHub -- many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do
nvidia-docker run -it ufoym/deepo bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
nvidia-docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash
This will make /host/data
from the host visible as /data
in the container, and /host/config
as /config
. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
You are now ready to begin your journey.
$ python
>>> import tensorflow
>>> print(tensorflow.__name__, tensorflow.__version__)
tensorflow 1.3.0
$ python
>>> import sonnet
>>> print(sonnet.__name__, sonnet.__path__)
sonnet ['/usr/local/lib/python3.5/dist-packages/sonnet']
$ python
>>> import torch
>>> print(torch.__name__, torch.__version__)
torch 0.2.0_3
$ python
>>> import keras
>>> print(keras.__name__, keras.__version__)
keras 2.0.8
$ python
>>> import mxnet
>>> print(mxnet.__name__, mxnet.__version__)
mxnet 0.11.0
$ python
>>> import cntk
>>> print(cntk.__name__, cntk.__version__)
cntk 2.2
$ python
>>> import chainer
>>> print(chainer.__name__, chainer.__version__)
chainer 3.0.0
$ python
>>> import theano
>>> print(theano.__name__, theano.__version__)
theano 0.10.0beta4+14.gb6e3768
$ python
>>> import lasagne
>>> print(lasagne.__name__, lasagne.__version__)
lasagne 0.2.dev1
$ python
>>> import caffe
>>> print(caffe.__name__, caffe.__version__)
caffe 1.0.0
$ caffe --version
caffe version 1.0.0
$ th
│ ______ __ | Torch7
│ /_ __/__ ________/ / | Scientific computing for Lua.
│ / / / _ \/ __/ __/ _ \ | Type ? for help
│ /_/ \___/_/ \__/_//_/ | https://github.com/torch
│ | http://torch.ch
│
│th>
. | modern-deep-learning | dl-docker | jupyter-deeplearning | Deepo |
---|---|---|---|---|
ubuntu | 16.04 | 14.04 | 14.04 | 16.04 |
cuda | :x: | 8.0 | 6.5-8.0 | 8.0 |
cudnn | :x: | v5 | v2-5 | v6 |
theano | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
tensorflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
sonnet | :x: | :x: | :x: | :heavy_check_mark: |
pytorch | :x: | :x: | :x: | :heavy_check_mark: |
keras | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
lasagne | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
mxnet | :x: | :x: | :x: | :heavy_check_mark: |
cntk | :x: | :x: | :x: | :heavy_check_mark: |
chainer | :x: | :x: | :x: | :heavy_check_mark: |
caffe | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
torch | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |