The value of “max locked memory” has been modified at the operating system level, but LSF still returns the original value.
Symptoms before updating max locked memory
[user1@cluster-h00 ~]$ bsub -q myQueue -W 120:00 -n 16 -P myProjectGroup -m compute-node1 -I ulimit -a Job <32400> is submitted to default queue <normal>. <<Waiting for dispatch ...>> <<Starting on compute-node1>> core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 1027790 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 4096 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 1027790 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
To resolve this issue,
# badmin hshutdown # badmin hstartup
[user1@cluster-h00 ~]$ bsub -q gpgpu -m compute-node1 -I ulimit -a Job <32490> is submitted to queue <gpgpu>. <<Waiting for dispatch ...>> <<Starting on compute-node1>> core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 515133 max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 4096 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 515133 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
References:
The Linux Cluster 