How to install CUDA and cuDNN in Linux?

As AI developers we often use a platform like Google Colab to test our prototype where CUDA and cuDNN are already configured. To run development processes such as training large datasets and running GPU-intensive services we must need the Device/Server to have CUDA and cuDNN configured.

A few months ago, Sanjay Chamlagain, my team lead, and I acquired an NVIDIA RTX 3090 Ti GPU to configure CUDA and cuDNN. It becomes challenging for us to configure CUDA on the GPU for the first time. After so many trials and test we finally figured out the way to install CUDA and configure cuDNN in Linux. In this blog, I will describe the verified way to install CUDA and cuDNN in Linux with the output in each step so that you will not stuck while following this tutorial.

Before starting CUDA configuration, you must have a compatible NVIDIA Driver installed in your system. So, let’s check the driver version supported by GPU.

ubuntu-drivers devices

Output:

In the above output, you will get the list of drivers supported for your GPU. This list may not same on your device because you might have a different GPU than mine. Before selecting the driver check the Nvidia driver and CUDA compatibility and then select and install the driver that is compatible with the CUDA version that you have required. You can check Nvidia driver and CUDA compatibility from the following links.

• https://docs.nvidia.com/deploy/cuda-compatibility/index.html
• https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

In my case, I want to use bothCUDA 10.X AND CUDA 11.X for my GPU-related computing because I used these versions frequently for my GPU-intensive applications before and will use them in future too. So, after checking the Nvidia driver and CUDA compatibility, it is found that the driver version >= 450.80.02 is compatible with both CUDA 10.X and CUDA 11.X. Here, I am gonna install the Nvidia driver nvidia-driver-470-server from the list of drivers supported.

sudo apt-get install nvidia-driver-470-server

Now, nvidia-driver-470-server is installed on my computer. To check about the installed GPU driver, let’s restart the system and type the following command in your terminal.

nvidia-smi

Output:

• From nvidia-smi output, we knew that the Nvidia driver 470.182.03 is installed on our system.
• It will show the higher CUDA version having compatibility with the Nvidia driver and GPU memory status.

Note: If you already have another version of CUDA pre-installed and want a fresh installation then, you have to uninstall CUDA and check the compatibility of your Nvidia driver with the CUDA version that you want to install now. Follow this blog on How to remove CUDA completely from Linux? before starting the following steps.

Now, we have completed the Nvidia-driver installation, let's start to set up CUDA and cuDNN.

CUDA refers to Compute Unified Device Architecture. It is a parallel computing platform and provides a set of programming tools and API that allows general-purpose computing on Graphics Processing Units(GPUs). You can think CUDA library as a way to talk to NVIDIA GPUs. If you want to learn more about CUD then go to the official website of NVIDIA https://developer.nvidia.com/cuda-toolkit.

To install cuda-toolkit, we first need to download .run file from the NVIDIA cuda-toolkit archive https://developer.nvidia.com/cuda-toolkit-archive.In the archive you will get the list of cuda-toolkit as below.

From the above list, select the cuda-toolkit version according to your requirements. Here, I have selected cuda-toolkit 10.2. Now, select architecture, distribution and version on the basis of the OS you have and choose the installer type .runfile(local).

Now, go to the terminal and download .run file following the above instructions.

wget https://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda_10.2.89_440.33.01_linux.run

Once the download is completed, let's run .run file using the following command. Make sure you are in the path where .run file is located.

sudo sh cuda_10.2.89_440.33.01_linux.run

Now, you have to accept End User License Agreement(EULA).

Once you accepted the CUDA end-user agreement, you will see the following interface in your terminal.

Let’s unselect the driver checkbox since driver 470 is installed. You must unselect Driver because it removes the dependency of CUDA on to driver version otherwise we need to install the Nvidia driver 440.33.01 to install cuda-10.2. Once you have unselected the checkbox, you are ready to install cuda-toolkit.

This is output for cuda-10.2successful installation in my case.

As shown in the above output, we need to add the CUDA bin path and lib64 path to the environment variable. You can add environment variables by adding the following two lines to.bashrc file.

export PATH=“/usr/local/cuda-10.2/bin:$PATH”  
export LD_LIBRARY_PATH=“/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH”

We have completed the CUDA installation, let’s start to set up cuDNN.

cuDNN refers to NVIDIA CUDA Deep Neural Network, which is a GPU-accelerated library of primitives for deep neural networks. cuDNN is built on top of CUDA. It provides highly tuned implementations of routines arising frequently in DNN applications.

Now, we need to download the following three packages from cudnn-archive.

• cuDNN Runtime Library for Ubuntu18.04 (Deb)
• cuDNN Developer Library for Ubuntu18.04 (Deb)
• cuDNN Code Samples and User Guide for Ubuntu18.04 (Deb)

To download the packages go to https://developer.nvidia.com/rdp/cudnn-archive and log in. You are eligible to download the packages once you have enrolled in NVIDIA Developer Program.

Now, extract and install the above three packages using sudo dpkg -i <package_name>.

sudo dpkg -i libcudnn7_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-dev_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-doc_7.6.5.32-1+cuda10.2_amd64.deb

Then, change your current working path to /usr/src/cudnn_samples_v7/mnistCUDNNusing

cd /usr/src/cudnn_samples_v7/mnistCUDNN

Then, clean and compile using

sudo make clean && sudo make

After successful compilation, you will get the following output:

And now run:

./mnistCUDNN

See the following output, It displays the classification result and test status as Test Passed! if you have successfully configured cuDNN.

Here, we have successfully installed and configured both CUDA and cuDNN. Now, we are ready for GPU-accelerated computing.

In summary, We have successfully installed and configured the Nvidia driver, CUDA and cuDNN. I hope that this tutorial will help you to configure CUDA in your GPU. If it works for you please don’t forget to clap and follow me on medium. See you in the next blog!