Checking nproc limits


One of our Linux Compute Server was showing when a particular was attempting to login on.

failed to execute /bin/bash: resource temporarily unavailable

We suspected that nprocs limits have been breached by the particular user. I found this write-up https://blog.dbi-services.com/linux-how-to-monitor-the-nproc-limit-1/ very prescriptive of the issue I faced.

Extracting information via ps is not useful unless you use the “-L” to show threads, possibly LWP (light-weight process).

 

% ps h -LA -o user | sort | uniq -c | sort -n
1 chrony
1 dbus
1 libstoragemgmt
1 nobody
1 rpc
1 rpcuser
2 avahi
2 user3
2 postfix
3 colord
3 rtkit
4 user1
4 user2
7 polkitd
23 user4
31 user5
34 user6
361 user7
442 user8
556 gdm
563 user9
922 user10
16384 user11
3319 root

You can see that user11 has 16384 threads!

To dig down into what is happening to a selected user. We will use user2 since it has one of the fewest LWP to

% ps -o nlwp,pid,lwp,args -u user2 | sort -n
NLWP PID LWP COMMAND
1 272705 272705 sshd: user2@pts/12
1 273054 273054 sshd: user2@notty
1 273216 273216 /usr/libexec/openssh/sftp-server
1 273406 273406 -bash

nlwp – Number of LWP
lwp – Process of ID of the LWP.

To eliminate the offending user11’s thousands of threads

% pkill -KILL -u user11

References

  1. Linux: how to monitor the nproc limit
  2. How is the nproc hard limit calculated and how do we change the value on CentOS 7

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.