If you are using Nvidia hpc-x and encountering issues like the one below during your MPI Run
shm_open(file_name=/ucx_shm_posix_77de2cf3 flags=0xc2) failed: Permission denied
The error message indicates that the shared memory has no permission to be used, The permission of /dev/shm is found to be 755, not 777, causing the error. The issue can be resolved after the permission is changed to 777. To change and verify the changes:
% chmod 777 /dev/shm
% ls -ld /dev/shm
drwxrwxrwx 2 root root 40 Jul 6 15:18 /dev/sh