Run DeepLearning Project In Docker

真正的优秀正是不断学习带来的结果,不时刻保持敬畏,了解这个崭新的世界,被淘汰那是必然。

本文将讲解,如何在Docker中运行Tensorflow、 Pytorh等深度学习框架。

需要在宿主机上提前安装好驱动和cuda,驱动版本需要比较高。

安装Docker和Nvidia-docker

配置 Docker 源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 更新源
$ sudo apt update

# 启用HTTPS
$ sudo apt install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common

# 添加GPG key
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# 添加稳定版的源
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"

安装 Docker CE

此刻 Docker 版本需要 19.03,此后可能需要更新。

1
2
3
4
5
# 更新源
$ sudo apt update

# 安装Docker CE
$ sudo apt install -y docker-ce

如果这种方式安装失败,也有解决方案。 报错时屏幕上会显示下载失败的 deb 文件,想办法下载下来,然后挨个手动安装就好。

此刻我需要下载的是下面三个文件,此后更新为当时最新版本即可:

手动依次安装:

1
2
3
$ sudo dpkg -i containerd.io_1.2.6-3_amd64.deb
$ sudo dpkg -i docker-ce-cli_19.03.0~3-0~ubuntu-bionic_amd64.deb
$ sudo dpkg -i docker-ce_19.03.0~3-0~ubuntu-bionic_amd64.deb

验证 Docker CE

如果出现下面的内容,说明安装成功。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ sudo docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:2557e3c07ed1e38f26e389462d03ed943586f744621577a99efb77324b0fe535
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

For more examples and ideas, visit:
https://docs.docker.com/get-started/

安装 nvidia-container-toolkit

1
2
3
4
5
6
7
8
# 添加源
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

# 安装并重启docker
$ sudo apt update && sudo apt install -y nvidia-container-toolkit
$ sudo systemctl restart docker

使用 nvidia-container-toolkit

1
2
3
4
5
6
7
8
# 在官方CUDA镜像上测试 nvidia-smi
$ sudo docker run --gpus all nvidia/cuda:9.0-base nvidia-smi

# 启动支持双GPU的容器
$ sudo docker run --gpus 2 nvidia/cuda:9.0-base nvidia-smi

# 指定GPU 1,运行容器
$ sudo docker run --gpus device=0 nvidia/cuda:9.0-base nvidia-smi

参考链接

https://www.cnblogs.com/journeyonmyway/p/11234572.html

安装Tensorflow和Pytorch

Keras

keras依赖于tensor flow,安装好tensorflow就行。

Pull Tensorflow Docker Images

配置可以使用keras起调GPU的Docker环境。keras以 tensorflow-gpu 为后端, 在这个页面 找到符合自己版本的tensorflow-gpu, pull下来。

1
docker pull tensorflow/tensorflow:1.9.0-devel-gpu

Run Tensorflow Docker Container

1
sudo docker --runtime=nvidia run -p 8085:8085 -v /home/brl/docker_share:/data --name speech_emotion_recognition --gpus all -it tensorflow/tensorflow:1.9.0-devel-gpu /bin/bash

去runtime版

1
sudo run -p 8085:8085 -v /home/brl/docker_share:/data --name container --gpus all -it tensorflow/tensorflow:1.9.0-devel-gpu /bin/bash

参考链接

tensorflow for docker homepage

https://www.tensorflow.org/install/docker?hl=zh-cn#examples_using_gpu-enabled_images

Pytorch

Pull Pytorch Docker Images

配置可以使用Pytorch起调GPU的Docker环境。去这个页面,找到符合自己版本的pytorch-gpu, pull下来。

1
sudo docker pull pytorch/pytorch:0.4.1-cuda9-cudnn7-runtime

Run Pytorch Docker Container

1
sudo docker run --runtime=nvidia -p 8086:8086 -v /home/brl/docker_share:/data --name sound_event_detection --gpus=all -it pytorch/pytorch:0.4.1-cuda9-cudnn7-runtime /bin/bash

去runtime版

1
sudo docker run -p 8086:8086 -v /home/brl/docker_share:/data --name sound_event_detection --gpus=all -it pytorch/pytorch:0.4.1-cuda9-cudnn7-runtime /bin/bash

本博客所有文章除特别声明外,均采用 CC BY-SA 4.0 协议 ,转载请注明出处!