Deploying Docker with GPU support on Windows Subsystem for Linux
At Aotu.ai we develop BrainFrame, a deep learning video analysis platform designed to make smart AI video inference accessible to everyone. BrainFrame makes heavy use of tools such as Docker, docker-compose, and CUDA. These tools allow us to accelerate inference on the GPU, and make it faster and easier to make deterministic deployments.
Even though BrainFrame is primarily deployed on Linux machines, some users like having the option to run and/or deploy on Windows. Unfortunately, getting this set up is a little tricky. In this blog you’ll learn to use WSL with GPU support and also how to deploy docker-compose services on Windows.
During this blog post we will be working in the windows command line. We will also be using a linux shell once WSL is installed on the system. The new (and fancy!) Windows Terminal, which can be downloaded here, greatly simplifies this process. It’s a breeze to use, looks good, and you can open multiple shells within its tab system even if they are from different linux distros or different windows command line tools. It’s also open source and can be found on GitHub!
Let’s start with some system requirements. WSL2 can be installed on most Windows 10 systems. If you are on a personal device, you probably have Windows 10 Home; this also works on Windows 10 Pro and Server editions.
WSL2 is available for both x64 and ARM64 architecture Windows computers. For x64 machines, you must have Version 1903 or higher, along with Build 18362 or higher. For ARM64 machines, you must have Version 2004 or higher, along with Build 19041 or higher. If for some reason you don’t know what architecture your computer uses you can use PowerShell to check.
systeminfo | find “System Type”
PowerShell can be run straight from your Windows Start menu.
To check your Version and Build, you can click the Settings icon in the Start menu, and navigate to System > About. Your windows specs will be displayed at the bottom of the window. You can also press Windows + R, type winver, and hit OK.
Your specs will look similar to the below image. If the version/build is not high enough, try updating Windows to the latest version.
Another requirement is having hardware virtualization enabled in your computer’s BIOS/UEFI. Most computers usually have this enabled by default. You can check this by opening the Task Manager (right click on taskbar and click Task Manager, or press Shift + Ctrl + Esc). Under the Performance tab, it should show a “Virtualization” area with either Enabled or Disabled. This should say Enabled. If it does not, you will have to restart your computer and modify the BIOS/UEFI settings. Look for Virtualization Technology, VTx, or something similar and ensure it is enabled.
Enabling Windows Subsystem for Linux
Windows Subsystem for Linux is an optional feature on Windows machines. We’ll need to enable it along with the Virtual Machine Platform feature for WSL2 to work. To enable it, open an elevated (with admin) PowerShell and run these commands:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
This will enable WSL and the Virtual Machine platform.
Installing Linux Kernel Update
Running WSL2 requires an update to the Linux kernel package. Be sure to install the package that is associated with your system. It is an installer so just double-click to run it. The packages can be found here:
Setting WSL2 as the default version
Once your kernel is updated we need to switch the default WSL version from 1 to 2, you may run this command to set WSL2 as default.
wsl –set-default-version 2
It may notify you that you don’t have any distributions installed, but this is okay as we will install some later and should not matter for now. This simply ensures that all future distros will default to using WSL2 instead of WSL1.
Install your flavor of Linux
Once the above has been completed, you may install any version of Linux you like from the Microsoft Store. Open the Microsoft Store and type Linux to get a list of available distributions available to you. Popular distros are Ubuntu, Debian, and Kali. For brevity, we will be using Ubuntu 20.04 in this blog post.
Once installed, you can launch it from the store or the Start menu, where it will take you through the initial user setup for the machine. (This should only happen the first time you start each distro.)
Enabling your GPU on WSL2
GPU usage in WSL2 is (unfortunately) only available through the Windows Insider Program right now. You will first have to register for the program, which is thankfully fairly simple.
Once registered, follow the instructions on the Windows Insider website to get started. Make sure you choose the Dev Channel when selecting Insider settings during setup. Also make sure you check for updates in the Windows Update panel, and install any Insider updates.
After installing the Windows Insider features, and subscribing to the latest dev channel, you may install the GPU drivers found here to get started using GPU. Note that these drivers only work on distros that use glibc, like Ubuntu or Debian.
If you skipped some of the above steps because you already had WSL2 installed, make sure that your Linux kernel version is greater than or equal to 4.19.121. If not, be sure to update it as the insider preview only works with that version and up.
Setting up CUDA in WSL2
We will be using Ubuntu 20.04 for this installation, but you may use any glibc-based distro you please. Just follow these instructions but with your distro’s tools. For more info refer to NVIDIA’s guide here.We will set up the CUDA Network Repository, and then install CUDA itself on the system. First run these commands in your linux shell:
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 /" > /etc/apt/sources.list.d/cuda.list' sudo apt-get update
Once set up we can now install CUDA. Use this package:
sudo apt-get install cuda-toolkit-11-0
This will take a while to download and install, so go grab a snack
Using Docker with GPU in WSL2
With CUDA now installed on the system, our next step is to set up our workflow for Docker containers.
There is a Docker desktop app for Windows, which is a fabulous tool for running Docker containers. While it provides a really good user experience, it unfortunately does not have GPU support, so we won’t be able to use it. We will have to install Docker using an install script within our Linux shell like this:
curl https://get.docker.com | sh
Next we need to install the NVIDIA Container Toolkit. I am installing this under Ubuntu 20.04 but you may use whatever you like. We must set up the experimental nvidia-docker2 repository as this is what supports WSL2 at this time. To do so, run these commands:
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list curl -s -L https://nvidia.github.io/libnvidia-container/experimental/$distribution/libnvidia-container-experimental.list | sudo tee /etc/apt/sources.list.d/libnvidia-container-experimental.list
Once that is set up, we can now install the package:
sudo apt-get update sudo apt-get install nvidia-docker2
Once the installation is finished, you may open another WSL2 window and start the docker daemon to complete your install.
sudo service docker stop sudo service docker start
Running Docker CUDA containers in WSL2
Great! Now that your install is complete, you can now run GPU containers in WSL2!
You can try running “nvidia-smi” to see if WSL2 is able to see your GPU. That should look a little like this:
Here are a few examples from the NVIDIA docs:
docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
Your output should look like this:
Or we can run a Jupyter notebook!
docker run -it --gpus all -p 8888:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter
Which should look like this:
Congratulations! You now have GPU support in WSL2!
Permission Denied errors when trying to run docker containers
If you seem to be getting a permission denied error when running docker containers, that means that you group may have not been added to the container. If this is the case, run this command in your linux shell:
sudo usermod -aG docker your_group_here
Replace “your_group_here” with the username you used to set up the linux environment
Distro created with WSL1
Distro created with WSL1
wsl --list --verbose
Sometimes for unknown reasons, a distribution may be created under WSL1 instead of WSL2.
This will cause errors later. To fix this, open a powershell window, and run this command:
wsl --set-version Ubuntu-20.04 2
Replace “Ubuntu-20.04” with your desired repo.