If you have installed the CUDA Drivers and CUDA SDK using the NVIDIA CUDA Installation Guide for Linux. Look for Section 3.3.3 for RHEL 8 / Rocky 9

If after following instruction, you are still facing issues, you may want to consider the following

Step 1 – Blacklist nouveau.conf

$ vim /etc/modprobe.d/blacklist-nouveau.conf

blacklist nouveau
options nouveau modeset=0

Step 2 – Remove Nvidia driver installation

# dnf module remove --all nvidia-driver

Step 3 – Remove CUDA-Related Installation

sudo dnf remove "cuda*" "*cublas*" "*cufft*" "*cufile*" "*curand*" \
 "*cusolver*" "*cusparse*" "*gds-tools*" "*npp*" "*nvjpeg*" "nsight*"

Step 4 – Reboot

# shutdown -r now

Step 5 – Make sure your BIOS SecureBoot is off. You can check

mokutil --sb-state
SecureBoot disabled

References:

1. Forum – CentOS Stream 8: NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver