One of my users was facing the issues when running ANSYS-20.1 Fluent
Error: eval: unbound variable Error Object: c-flush
... ... ... libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs2 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs2 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 fluent_mpi.20.1.0: Rank 0:110: MPI_Init: ibv_create_qp(left ring) failed fluent_mpi.20.1.0: Rank 0:110: MPI_Init: probably you need to increase pinnable memory in /etc/security/limits.conf fluent_mpi.20.1.0: Rank 0:110: MPI_Init: ibv_ring_createqp() failed fluent_mpi.20.1.0: Rank 0:110: MPI_Init: Can't initialize RDMA device fluent_mpi.20.1.0: Rank 0:110: MPI_Init: Internal Error: Cannot initialize RDMA protocol ... ... ...
There were a few steps that we took to resolve the issues. Do look at the generated logs which will give lots of information. If you are using the PBS-Professional. your fluent script may look like this.
fluent 3ddp -g -t$nprocs -mpi=ibmmpi -cflush -ssh -lsf -cnf=$PBS_NODEFILE -pinfiniband -nmon -i $inputfile >& $PBS_JOBID.log 2>&1
- Do note that “-cflush” and that it should be placed after the -mpi flag. From our observation, if the -cflush is placed before -mpi, it seems to initiate the flush, but another error would be generated.
- Make sure you choose the right mpi parameter.