Basic Usage of Singularity (Part 1)

The article is taken from https://github.com/NIH-HPC/Singularity-Tutorial. I’m personally learning much from the systematic rich contents and below are my learning points……

Downloading Singularity

https://github.com/sylabs/singularity

Popular Sites

There are popular sites for Singularity Download that has pre-build containers

Downloading the Containers

% singularity pull library://godlovedc/funny/lolcow
INFO:    Downloading library image
89.2MiB / 89.2MiB [========================================================================================================================================================] 100 % 5.4 MiB/s 0s
WARNING: integrity: signature not found for object group 1
WARNING: Skipping container verification

Singularity File

The Singularity File has a .sif extension something like this

lolcow_latest.sif

Entering the Shell of Singularity

% singularity shell lolcow_latest.sif
Singularity> cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

You will notice the container is running Ubuntu…… although the Host OS could be different

Interesting Observations

a. The user remains the same inside and outside of the container.

Singularity> whoami
admin

b. The hostname remains the same inside and outside the container

Singularity> hostname
node1

c. Running the application within the containers

Singularity> cowsay moo
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

d. Exiting the Application

Singularity > exit

References:

  1. https://github.com/NIH-HPC/Singularity-Tutorial
  2. https://github.com/sylabs/singularity

How to configure NFS on CentOS 7

Step 1: Do a Yum Install

# yum install nfs-utils rpcbind

Step 2: Enable the Service at Boot Time

# systemctl enable nfs-server
# systemctl enable rpcbind
# systemctl enable nfs-lock     (it does not need to be enabled since rpc-statd.service  is static.)
# systemctl enable nfs-idmap    (it does not need to be enabled since nfs-idmapd.service is static.)

Step 3: Start the Services

# systemctl start rpcbind
# systemctl start nfs-server
# systemctl start nfs-lock
# systemctl start nfs-idmap

Step 4: Confirm the status of NFS

# systemctl status nfs

Step 5: Create a mount point

# mkdir /shared-data

Step 6: Exports the Share

# vim /etc/exports
/shared-data 192.168.0.0/16(rw,no_root_squash)

Step 7: Export the Share

# exportfs -rv

Step 8: Restart the NFS Services

# systemctl restart nfs-server

Step 9: Configure the Firewall

# firewall-cmd --add-service=nfs --zone=internal --permanent
# firewall-cmd --add-service=mountd --zone=internal --permanent
# firewall-cmd --add-service=rpc-bind --zone=internal --permanent

References:

  1. How to configure NFS in RHEL 7
  2. What firewalld services should be active on an NFS server in RHEL 7?

CentOS Project is shifting to CentOS Stream

The CentOS project recently announced a shift in strategy for CentOS.

  • It will be shifting to an upstream build (testing patches and updates before inclusion in the upstream vendor).
  • Additionally, support for CentOS Linux 8 has been cut short, from May 31, 2029 to December 31, 2021.
  • CentOS 7 will continue to be supported as a downstream version of RHEL 7 till Jun 2024.

 

 

Where do we go from here? We can look at Rocky Linux. Rocky Linux aims to function as a downstream build as CentOS had done previously, building releases after they have been added by the upstream vendor, not before. Rocky Linux is led by Gregory Kurtzer, founder of the CentOS project.

 

Listing processes for a specific user

Using htop to list users. Which is one of my favourite.

% top -U user1

pstree which displays a tree of processes and can include parents and child processes which make it easier to understand.

% pstree -l -a -p -s user1


where
-l : Long format
-a : Show command line args
-p : Display Linux PIDs
-s : See parents of the selected process

pgrep look up or signal processes based on name and other attributes

% pgrep -l -u user1

References:

  1. Linux list processes by user names

mount.nfs: requested NFS version or transport protocol is not supported

If you have encountered issues like

mount.nfs: requested NFS version or transport protocol is not supported

OR

mount.nfs4: Protocol not supported

To resolve this

Mount with NFS version 3 (with 4 verbose flags)

% mount -vvvv -t nfs -o vers=3 nfs-server:/share /mnt/nfs

References:

  1. Error “mount.nfs: requested NFS version or transport protocol is not supported” when attempting to mount an NFS share on Red Hat Enterprise Linux 6

Using nfsstat to troubleshoot NFS performance issues

The write-up is taken from RedHat Using nfsstat and nfsiostat to troubleshoot NFS performance issues on Linux

NFS relies on the existing network infrastructure, any glitches on the network may affect the performance of the connection. One of the tools that can be used is nfsstat

% yum install nfs-utils

The nfsstat command

The nfsstat command displays statistical information about the NFS and Remote Procedure Call (RPC) interfaces to the kernel.

On Server Side,

% nfsstat -s
Server rpc stats:
calls badcalls badclnt badauth xdrcall
107310012 0 0 0 0

The most important field to check is the badcalls, which represents the total number of calls rejected by the RPC layer. When the badcalls is greater than 0, than the underlying network needs to be checked, as there might be latency.

 

On NFS Client Side,

% nfsstat -c
Client rpc stats:
calls retrans authrefrsh
23158 0 23172

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 7237 31% 7 0% 1443 6% 7874 34% 11 0%
read write create mkdir symlink mknod
578 2% 4548 19% 585 2% 1 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
0 0% 0 0% 0 0% 0 0% 0 0% 51 0%
fsstat fsinfo pathconf commit
25 0% 10 0% 5 0% 781 3%

The client is doing well as it has relatively few retransmission requests. If you are encountering excessive retransmissions, you may want to adjust data transfer buffer sizes, which are specified by the mount command options rsize and size.

 

Check for dropped packets

Check dropped packet by running the following command on both the server and the client:

% nfsstat -o net
Client packet stats:
packets udp tcp tcpconn
0 0 0 0

Yum History and Using Yum to Roll Back updates

Getting History of YUM actions

Point 1: List Yum actions list

(base) [root@hpc-node1 ~]# yum history list
Loaded plugins: fastestmirror, langpacks
ID | Login user | Date and time | Action(s) | Altered
-------------------------------------------------------------------------------
42 | root <root> | 2020-06-30 11:36 | I, U | 3
41 | 12345 | 2020-06-04 16:43 | Install | 2
40 | 12345 | 2020-06-04 16:26 | I, O, U | 878 E<
39 | 12345 | 2020-03-20 13:54 | Install | 1 >
38 | 12345 | 2020-03-20 13:46 | Install | 1
37 | 12345 | 2020-03-20 13:45 | Install | 2
36 | 12345 | 2020-03-20 13:44 | Install | 1
35 | 12345 | 2020-03-20 13:43 | Install | 28
34 | 12345 | 2020-03-20 12:52 | Update | 4
33 | 12345 | 2020-03-20 12:51 | Install | 1
32 | 12345 | 2020-03-20 12:43 | Update | 1
31 | 12345 | 2020-03-20 11:10 | Install | 1
30 | 12345 | 2020-03-20 10:53 | Install | 2
29 | 12345 | 2020-02-20 14:54 | Install | 1
28 | 12345 | 2020-01-23 16:10 | I, U | 2
27 | 12345 | 2020-01-15 15:03 | Update | 15
26 | 12345 | 2020-01-15 15:03 | I, U | 18
25 | 12345 | 2020-01-15 15:03 | Update | 3
24 | 12345 | 2019-11-07 11:00 | Install | 1 <
23 | 12345 | 2019-09-23 12:47 | Install | 1 >

Point 2: Undo the Yum action.

(base) [root@hpc-node1 boot]# yum history undo 42
Loaded plugins: fastestmirror, langpacks
Undoing transaction 42, from Tue Jun 30 11:36:18 2020
.....
.....
Running transaction
  Installing : ntpdate-4.2.6p5-29.el7.centos.x86_64                                                                                                                                             1/4
  Erasing    : ntp-4.2.6p5-29.el7.centos.2.x86_64                                                                                                                                                       2/4
  Erasing    : autogen-libopts-5.18-5.el7.x86_64                                                                                                                                                        3/4
  Cleanup    : ntpdate-4.2.6p5-29.el7.centos.2.x86_64                                                                                                                                                   4/4
  Verifying  : ntpdate-4.2.6p5-29.el7.centos.x86_64                                                                                                                                                     1/4
  Verifying  : ntp-4.2.6p5-29.el7.centos.2.x86_64                                                                                                                                                       2/4
  Verifying  : autogen-libopts-5.18-5.el7.x86_64                                                                                                                                                        3/4
  Verifying  : ntpdate-4.2.6p5-29.el7.centos.2.x86_64                                                                                                                                                   4/4

Removed:
  autogen-libopts.x86_64 0:5.18-5.el7                              ntp.x86_64 0:4.2.6p5-29.el7.centos.2                              ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2

Installed:
  ntpdate.x86_64 0:4.2.6p5-29.el7.centos

Complete!

Point 3 – Check on the history of a particular package

yum history list ntpdate
Loaded plugins: fastestmirror, langpacks
ID     | Login user               | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
    43 | root               | 2020-07-06 11:31 | D, E           |    3
    42 | root               | 2020-06-30 11:36 | I, U           |    3
    40 | root               | 2020-06-04 16:26 | I, O, U        |  878 EE
     2 | root               | 2019-07-09 09:31 | I, U           | 1029 EE