How to Set Up Nvidia Drivers (Open Kernel) and CUDA Toolkit on Ubuntu 24.04

Purpose:

This guide provides step-by-step instructions on how to perform a clean installation of the NVIDIA Open Kernel Driver and configure the CUDA Toolkit environment on Ubuntu 24.04 LTS.

Target Audience:

For the users who want to use NVIDIA GPU on Ubuntu 24.04

Driver Compatibility & Supported Devices

Starting with driver version R560, NVIDIA has transitioned to Open Kernel Modules as the default for modern GPU architectures. 

GPU Architecture	Representative Products	Driver Requirement
Blackwell	RTX 5090, H200, RTX PRO 6000	Mandatory Open Driver (Proprietary not supported)
Hopper / Ada Lovelace	H100, H200, L40S, L4, HGX H100, RTX 6000 Ada, A100, A40, A10, Tesla T4, RTX A6000, Quadro RTX 8000	Recommended Open Driver (Proprietary legacy supported)
Volta / Pascal / Maxwell	Tesla V100, P100, K80, M40	Legacy Proprietary Only (Do NOT use this guide)

Procedure

Step 1: Purge Existing NVIDIA Drivers

To avoid dependency conflicts, completely remove any previously installed NVIDIA drivers from your system.

sudo apt purge "^nvidia-.*"

Step 2: Install the NVIDIA Open Driver

Nvidia recommends the Open Kernel Modules for Turing architecture and newer GPUs on Linux:

sudo apt update && sudo apt upgrade -y

sudo apt install nvidia-driver-580-open

Reboot your system to load the new kernel modules:

sudo reboot

You can find the cuda and driver compatibility on this page:

https://docs.nvidia.com/deploy/cuda-compatibility/minor-version-compatibility.html

Note on Secure Boot: If your system has Secure Boot enabled, Ubuntu will prompt you to create a password for the MOK (Machine Owner Key). After rebooting, you must enter the MOK management screen . Otherwise, the driver will not be loaded.

Step 3: Download and Install CUDA Toolkit

1. Go to the official NVIDIA Developer Network page to get the tailored installation commands for Ubuntu 24.04:

   NVIDIA CUDA Toolkit Downloads 

2. Select the architecture (x86_64), distribution (Ubuntu), version (24.04), and installer type (deb (local)).

3. Execute the sequence of commands provided on the NVIDIA page in your terminal to complete the installation.

Step 4: Verify the CUDA Installation Path

Before setting up your environment variables, verify the exact version of the CUDA toolkit that was installed.

1. List the contents of the /usr/local/ directory:

   ls /usr/local/

Step 5: Configure Environment Variables

To allow your system and development frameworks to locate the CUDA binaries and libraries, you must append them to your environment paths.

1. Open your .bashrc configuration file using the Nano text editor:

   nano ~/.bashrc

2. Scroll to the very bottom of the file and paste the following lines  (If you have multiple CUDA versions installed and need to use a specific one (e.g., 13.2), replace cuda with your actual version folder name):

   export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
   export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

3. Source the file to apply the changes immediately to your current terminal session:

   source ~/.bashrc

Verification

To ensure everything has been correctly configured, run the following verification commands:

• Check Driver Status:

  nvidia-smi

• Check CUDA Compiler Version:

  nvcc -V