

How to Install NVIDIA Container Toolkit

Learn how to install the NVIDIA Container Toolkit on Ubuntu with our step-by-step instructions. Unlock the power of GPU-accelerated containers today.

What's NVIDIA Container Toolkit?

The NVIDIA Container Toolkit is a set of tools and libraries that enable GPU-accelerated Docker containers. It allows you to leverage NVIDIA GPUs within containers by providing the necessary drivers and runtime components. This toolkit is essential for running CUDA applications, machine learning workloads, and other GPU-accelerated tasks in Docker containers.

Nvidia-Docker2 vs Nvidia-Container-Toolkit

Nvidia-Docker2: This package was the original way to enable GPU support in Docker containers. It included the necessary runtime components and configurations to integrate NVIDIA GPUs with Docker. However, it has been deprecated in favor of the more flexible nvidia-container-toolkit.

Nvidia-Container-Toolkit: This is the current and recommended way to enable NVIDIA GPU support in Docker containers. It is more modular and decouples the runtime from the Docker package, allowing for greater flexibility and easier updates. This toolkit allows you to use the --gpus flag in Docker to specify GPU resources.

Key Features of NVIDIA Container Toolkit

NVIDIA GPU Support: Access NVIDIA GPUs from within Docker containers.

CUDA Toolkit Integration: Provides necessary components to run CUDA applications.

Compatibility: Works with different versions of Docker and various NVIDIA GPUs.

Prerequisites

CUDA-capable GPU: Ensure you have an NVIDIA GPU installed.

Docker: Docker must be installed on your system. You can install Docker by following the official guide here.

NVIDIA Driver: Ensure that the appropriate NVIDIA driver is installed on your system. You can check the installation with the following command: nvidia-smi.

Installing the NVIDIA Container Toolkit

Here’s a step-by-step guide to install the NVIDIA Container Toolkit on Ubuntu:

Step 1. Configure the production repository:

$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Step 2. Update the packages list from the repository:

$ sudo apt-get update

Step 3. Install the NVIDIA Container Toolkit packages:

$ sudo apt-get install -y nvidia-container-toolkit

Step 4. Configure the container runtime by using the nvidia-ctk command:

$ sudo nvidia-ctk runtime configure --runtime=docker

The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.

Step 5. Restart the Docker daemon:

$ sudo systemctl restart docker

Step 6. Test the Installation

Run a test container:

$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
$ sudo docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi

This command should output information about your GPU, confirming that the container has access to the GPU.