System Administration

by

Installing Environment Modules on Rocky Linux 8.5

Step 1: Download the modules packages

You can download the latest version of Modules Environment from http://modules.sourceforge.net/. The current version is 5.1

% dnf install tcl tcl-devel
% tar -zxvf modules-5.1.0.tar.gz
% cd modules-5.1.0
% ./configure --prefix=/usr/local/Modules \
--modulefilesdir=/usr/local/Modules/modulefiles
$ make && make install

By default, /usr/local/Modules/modulefiles will be setup as the default directory containing modulefiles. –modulefilesdir option enables to change this directory location.

References:

  1. Installing Modules on Unix
by

Managing of Roaming Users’ Home Directories with Systemd-Homed

This article can be taken from OpenSource.com titled “Manage Linux users’ home directories with systemd-homed“

Image By: OpenSource.com

The systemd-homed service supports user account portability independent of the underlying computer system. A practical example is to carry around your home directory on a USB thumb drive and plug it into any system which would automatically recognize and mount it. According to Lennart Poettering, lead developer of systemd, access to a user’s home directory should not be allowed to anyone unless the user is logged in. The systemd-homed service is designed to enhance security, especially for mobile devices such as laptops. It also seems like a tool that might be useful with containers.

This objective can only be achieved if the home directory contains all user metadata. The ~/.identity file stores user account information, which is only accessible to systemd-homed when the password is entered. This file holds all of the account metadata, including everything Linux needs to know about you, so that the home directory is portable to any Linux host that uses systemd-homed. This approach prevents having an account with a stored password on every system you might need to use.

The home directory can also be encrypted using your password. Under systemd-homed, your home directory stores your password with all of your user metadata. Your encrypted password is not stored anywhere else thus cannot be accessed by anyone. Although the methods used to encrypt and store passwords for modern Linux systems are considered to be unbreakable, the best safeguard is to prevent them from being accessed in the first place. Assumptions about the invulnerability of their security have led many to ruin.

This service is primarily intended for use with portable devices such as laptops. Poettering states, “Homed is intended primarily for client machines, i.e., laptops and thus machines you typically ssh from a lot more than ssh to, if you follow what I mean.” It is not intended for use on servers or workstations that are tethered to a single location by cables or locked into a server room.

The systemd-homed service is enabled by default on new installations—at least for Fedora, which is the distro that I use. This configuration is by design, and I don’t expect that to change. User accounts are not affected or altered in any way on systems with existing filesystems, upgrades or reinstallations that keep the existing partitions, and logical volumes.

Manage Linux users’ home directories with systemd-homed (OpenSource.com)

For more Read-Up, do take a look at “Manage Linux users’ home directories with systemd-homed“

by

Checking Disk Usage within the subfolders but avoid mount-point

If you need to check Usage, but you wish to avoid the mount-point, you can use the command

[root@hpc-hn /]# du -h -x -d 1
48M     ./etc
552M    ./root
11G     ./var
1.1G    ./tmp
11G     ./usr
0       ./media
0       ./mnt
4.8G    ./opt
0       ./srv
0       ./install
0       ./log
0       ./misc
0       ./net
0       ./server_priv
0       ./ProjectSpace
0       ./media1
0       ./media2
28G     .
  • -h refers to human-readable
  • -d refers to depth level. By default, it is 0 which is the same as summarize
  • -x skip directories on different file systems
by

Creating a Self-Signed Certificate on RHEL

You can create your own self-signed certificate. Note that a self-signed certificate does not provide the security guarantees of a CA-signed certificate.

Generating a Key

Taken from RHEL Administration Guide 25.6. GENERATING A KEY and Creating a Self-Signed Certificate

Step 1: Clean up fake key and certificate

Go to /etc/httpd/conf/ directory. Remove the fake key and certificate that were generated during the installation

# cd /etc/httpd/conf/
# rm ssl.key/server.keyrm ssl.crt/server.crt

Step 2: Create your own Random Key

Go to usr/share/ssl/certs/ and generate key

# cd /usr/share/ssl/certs/
# make genkey

Your system displays a message similar to the following:

mask 77 ; \
/usr/bin/openssl genrsa -des3 1024 > /etc/httpd/conf/ssl.key/server.key
Generating RSA private key, 1024 bit long modulus
.......++++++
................................................................++++++
e is 65537 (0x10001)
Enter pass phrase:

You now must enter in a passphrase. For security reason, it should contain at least eight characters, include numbers and/or punctuation, and it should not be a word in a dictionary.

Re-type the passphrase to verify that it is correct. Once you have typed it in correctly, /etc/httpd/conf/ssl.key/server.key, the file containing your key, is created.

Note that if you do not want to type in a passphrase every time you start your secure server, you must use the following two commands instead of make genkey to create the key.

# /usr/bin/openssl genrsa 1024 > /etc/httpd/conf/ssl.key/server.key

Then, use the following command to make sure the permissions are set correctly for the file:

# chmod go-rwx /etc/httpd/conf/ssl.key/server.key

After you use the above commands to create your key, you do not need to use a passphrase to start your secure server.

* The server.key file should be owned by the root user on your system and should not be accessible to any other user. Make a backup copy of this file and keep the backup copy in a safe, secure place. You need the backup copy because if you ever lose the server.key file after using it to create your certificate request, your certificate no longer works and the CA is not able to help you. Your only option is to request (and pay for) a new certificate.

Creating a Self-Signed Certificate

Once you have a key, make sure you are in the /usr/share/ssl/certs/ directory, and type the following command:

# /usr/share/ssl/certs/make testcert

The following output is shown and you are prompted for your passphrase (unless you generated a key without a passphrase):

umask 77 ; \
/usr/bin/openssl req -new -key -set_serial num /etc/httpd/conf/ssl.key/server.key  
-x509 -days 365 -out /etc/httpd/conf/ssl.crt/server.crt
Using configuration from /usr/share/ssl/openssl.cnf
Enter pass phrase:

Next, you are asked for more information. The computer’s output and a set of inputs looks like the following (provide the correct information for your organization and host):

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a
DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:SG

After you provide the correct information, a self-signed certificate is created in /etc/httpd/conf/ssl.crt/server.crt. Restart the secure server after generating the certificate with following the command:

# /sbin/service httpd restart
by

rsync and write failed with No Space left on Device (28)

If you run an rsync such as this command 

% rsync -lH -rva --no-inc-recursive --progress gromacs remote_server:/usr/local

and you encountered something like this

% rsync: write failed on "/usr/local": No space left on device (28)

After checking that the source and destination have sufficient space, you are still encountering the issue, you may want to put this parameter in “–inplace”. According to the rsync man page. “This option changes how rsync transfers a file when its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file.”

WARNING: you should not use this option to update files that are being accessed by others, so be careful when choosing to use this for a copy. For more information, do take a look at https://download.samba.org/pub/rsync/rsync.html 

% rsync -lH -rva --inplace --no-inc-recursive --progress gromacs remote_server:/usr/local
by

UDP Tuning to maximise performance

There is a interesting article how your UDP traffic can maximise performance with a few tweak. The article is taken from UDP Tuning

The most important factors as mentioned in the article is

  • Use jumbo frames: performance will be 4-5 times better using 9K MTUs
  • packet size: best performance is MTU size minus packet header size. For example for a 9000Byte MTU, use 8972 for IPV4, and 8952 for IPV6.
  • socket buffer size: For UDP, buffer size is not related to RTT the way TCP is, but the defaults are still not large enough. Setting the socket buffer to 4M seems to help a lot in most cases
  • core selection: UDP at 10G is typically CPU limited, so its important to pick the right core. This is particularly true on Sandy/Ivy Bridge motherboards.

Do take a look at the article UDP Tuning

by

Understanding Load Average in Linux

Taken from RedHat Article “What is the relation between I/O wait and load average?” I have learned quite a bit on this article.

Linux, unlike traditional UNIX operating systems, computes its load average as the average number of runnable or running processes (R state), and the number of processes in uninterruptable sleep (D state) over the specified interval. On UNIX systems, only the runnable or running processes are taken into account for the load average calculation.

On Linux the load average is a measurement of the amount of “work” being done by the machine (without being specific as to what that work is). This “work” could reflect a CPU intensive application (compiling a program or encrypting a file), or something I/O intensive (copying a file from disk to disk, or doing a database full table scan), or a combination of the two.

In the article, you can determine whether the high load average is the result processes in the running state or uninterruptible state,

I like this script…… that was written in the knowledgebase. The script show the running, blocked and runnin+blocked. 

[user@node1 ~]$ while true; do echo; uptime; ps -efl | awk 'BEGIN {running = 0; blocked = 0} $2 ~ /R/ {running++}; $2 ~ /D/ {blocked++} END {print "Number of running/blocked/running+blocked processes: "running"/"blocked"/"running+blocked}'; sleep 5; done

 23:45:52 up 52 days,  7:06, 22 users,  load average: 1.40, 1.26, 1.02
Number of running/blocked/running+blocked processes: 3/1/4

 23:45:57 up 52 days,  7:06, 22 users,  load average: 1.45, 1.27, 1.02
Number of running/blocked/running+blocked processes: 2/0/2

 23:46:02 up 52 days,  7:06, 22 users,  load average: 1.41, 1.27, 1.02
Number of running/blocked/running+blocked processes: 1/1/2

 23:46:07 up 52 days,  7:07, 22 users,  load average: 1.46, 1.28, 1.03
Number of running/blocked/running+blocked processes: 2/0/2

 23:46:12 up 52 days,  7:07, 22 users,  load average: 1.42, 1.27, 1.03
Number of running/blocked/running+blocked processes: 2/0/2

 23:46:17 up 52 days,  7:07, 22 users,  load average: 1.55, 1.30, 1.04
Number of running/blocked/running+blocked processes: 2/0/2

 23:46:22 up 52 days,  7:07, 22 users,  load average: 1.51, 1.30, 1.04
Number of running/blocked/running+blocked processes: 1/1/2

 23:46:27 up 52 days,  7:07, 22 users,  load average: 1.55, 1.31, 1.05
Number of running/blocked/running+blocked processes: 2/0/2

 23:46:32 up 52 days,  7:07, 22 users,  load average: 1.62, 1.33, 1.06
Number of running/blocked/running+blocked processes: 2/1/3

 23:46:38 up 52 days,  7:07, 22 users,  load average: 1.81, 1.38, 1.07
Number of running/blocked/running+blocked processes: 1/1/2

 23:46:43 up 52 days,  7:07, 22 users,  load average: 1.66, 1.35, 1.07
Number of running/blocked/running+blocked processes: 1/0/1

 23:46:48 up 52 days,  7:07, 22 users,  load average: 1.53, 1.33, 1.06
Number of running/blocked/running+blocked processes: 1/0/1

Another useful way to typical top output when the load average is high (filter the idle/sleep status tasks with i). So the high load average is because lots of sendmail tasks are in D status. They may be waiting either for I/O or network.

op - 13:23:21 up 329 days,  8:35,  0 users,  load average: 50.13, 13.22, 6.27
Tasks: 437 total,   1 running, 435 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.1%us,  1.5%sy,  0.0%ni, 93.6%id,  4.5%wa,  0.1%hi,  0.2%si,  0.0%st
Mem:  34970576k total, 24700568k used, 10270008k free,  1166628k buffers
Swap:  2096440k total,        0k used,  2096440k free, 11233868k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
11975 root      15   0 13036 1356  820 R  0.7  0.0   0:00.66 top                
15915 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15918 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15920 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15921 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15922 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15923 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15924 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15926 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15928 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15929 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15930 root      17   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail           
15931 root      18   0  5312  872   80 D  0.0  0.0   0:00.00 sendmail

References:

  1. What is the relation between I/O wait and load average?

by

Error “Too many files open” on CentOS 7

If you are encountering Error messages during login with “Too many open files” and the session gets terminated automatically, it is because the open file limit for a user or system exceeds the default setting and  you may wish to change it

@ System Levels

To see the settings for maximum open files,

# cat /proc/sys/fs/file-max
55494980

This value means that the maximum number of files all processes running on the system can open. By default this number will automatically vary according to the amount of RAM in the system. As a rough guideline it will be about 100,000 files per GB of RAM.

 

To override the system wide maximum open files, as edit the /etc/sysctl.conf

# vim /etc/sysctl.conf
 fs.file-max = 80000000

Activate this change to the live system

# sysctl -p

@ User Level

To see the setting for maximum open files for a user

# su - user1
$ ulimit -n
1024

To change the setting, edit the /etc/security/limits.conf

$ vim /etc/security/limits.conf
user - nofile 2048

To change for all users

* - nofile 2048

This set the maximum open files for ALL users to 2048 files. These settings will require a reboot to activate.

References:

  1. How to correct the error “Too many files open” on Red Hat Enterprise Linux