Getting Useful Information on CPU and Configuration

Point 1. lscpu

To install

yum install util-linux

lscpu – (Print out information about CPU and its configuration)

[user1@myheadnode1 ~]$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz
Stepping: 4
CPU MHz: 3200.000
BogoMIPS: 6400.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 25344K
NUMA node0 CPU(s): 0-7,16-23
NUMA node1 CPU(s): 8-15,24-31
Flags: fpu .................

Point 2: hwloc-ls

To install hwloc-ls

yum install hwloc

hwloc – (Prints out useful information about the NUMA locality of devices and general hardware locality information)

[user1@myheadnode1 ~]# hwloc-ls
Machine (544GB total)
NUMANode L#0 (P#0 256GB)
Package L#0 + L3 L#0 (25MB)
L2 L#0 (1024KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
PU L#0 (P#0)
PU L#1 (P#16)
L2 L#1 (1024KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
PU L#2 (P#1)
PU L#3 (P#17)
L2 L#2 (1024KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2
PU L#4 (P#2)
PU L#5 (P#18)
L2 L#3 (1024KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3
PU L#6 (P#3)
PU L#7 (P#19)
L2 L#4 (1024KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4
PU L#8 (P#4)
PU L#9 (P#20)
L2 L#5 (1024KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5
PU L#10 (P#5)
PU L#11 (P#21)
L2 L#6 (1024KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6
PU L#12 (P#6)
PU L#13 (P#22)
L2 L#7 (1024KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7
PU L#14 (P#7)
PU L#15 (P#23)
.....
.....
.....

Point 3 – Check whether the Boost is on for AMD

Print out if CPU boost is on or off

cat /sys/devices/system/cpu/cpufreq/boost
1

References:

  1. Tuning Guard for AMD EPYC (pdf)

Checking the Limits an application is imposed during run

If you wish to look at a specific application limits during run, you can do the following

pgrep fortcom
12345

* I used for fortcom, but it could be any application you wish to take a look.

cat /proc/12345/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 4096 2190327 processes
Max open files 1024 4096 files
Max locked memory unlimited unlimited bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 2190327 2190327 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

* You can take a look that there is no limits to Max locked Memory and Max file locks are unlimited.

Another way to calculate shared memory swapping

Using ipcs utlities to find out information on shared memory utilisation which can be useful for analysing the performance of the system. Let’s says you want to measure how much memory has been swapped.

ipcs -mu
------ Shared Memory Status --------
segments allocated 55
pages allocated 6655333
pages resident  5661034
pages swapped   947522
Swap performance: 0 attempts     0 successes

where
-m is “information about active shared memory segments”
-u is “Show status summary”

You would need PAGE Memory

getconf PAGESIZE
4096

To provide us with the information in MB

echo "$((947522*4096/1024/1024)) MB"
3701 MB

 

Using strace to detect df hanging issues on NFS

strace is a wonderful tool to trace system calls and signals

I was hanging issues whenever I do a “df” and I was curious which file system is calling issues

strace df
.....
.....
stat("/run/user/1304561586", {st_mode=S_IFDIR|0700, st_size=40, ...}) = 0
stat("/run/user/17132623", {st_mode=S_IFDIR|0700, st_size=40, ...}) = 0
stat("/run/user/17149581", {st_mode=S_IFDIR|0700, st_size=40, ...}) = 0
stat("/run/user/1304565184", {st_mode=S_IFDIR|0700, st_size=60, ...}) = 0
stat("/scratch",

It is obvious that /scratch file hang immediately after being launched.

Using Find and Tar Together to Backup and Archive

Point 1: If you wish to find files in a single folder and tar them into gzip-compressed archive. You can use a one-liner to do it.

find -maxdepth 1 -name '*.sh' | tar czf script.tgz -T -

“-maxdepth” refers to the current depth of 1 or current directory

“-T -” causes tar to read its list from a file rather than the command line. The “-” means standard input and output.

You should have a file is script.tgz

 

Point 2: If you wish to find files in a single folder and tar them into bzip2-compressed archive.

find -maxdepth 1 -name '*.sh' | tar cjf script.tgz -T -

Checking Disk Usage within the subfolders

I like this command which I often use to look into the dish usages at the sub folder level to check for large usages

du -h -d 1
1.3M    ./Espresso-BEEF
65M     ./MATLAB
478M    ./Abaqus
10G     ./COMSOL
8.3M    ./Gaussian2
316K    ./scripts
4.9M    ./NB07
647M    ./pytorch-GAN
31M     ./Gaussian
12G     .

where

  • -h refers to human-readable
  • -d refers to depth level. By default, it is 0 which is the same as summarize

Using BASH to read Keyboard Input

Sometimes, you may want to write a simple BASH menu system to help non-Linux users. Here is a sample sample.

#!/bin/bash
clear
echo "

Welcome to GCluster User Help Menu System
===============================================

Please select:
---------------------------------------------------
1. Display Current Logged Session(s)
2. Display Current Home Space Utilisation
3. Check Running Process(s)
4. Check my Project Funds
5. Check for ABAQUS Licenses
6. Check for MATLAB Distributed Compute Licenses
7. Check for ANSYS Licenses
8. List running jobs from Project Members
0. Quit
---------------------------------------------------
"

read -p "Enter selection [0-8] > "

if [[ $REPLY =~ ^[0-8]$ ]]; then
        if [[ $REPLY == 0 ]]; then
                echo "Program Terminated."
                exit
        fi
        if [[ $REPLY == 1 ]]; then
                echo "Logged session: "
                w |grep $USER
                exit
        fi
        if [[ $REPLY == 2 ]]; then
                echo "Home Space Utilisation ($USER):"
                echo "(Please be patient.... It will take a while)"
                du -sh $HOME
                exit
        fi
        if [[ $REPLY == 3 ]]; then
                echo "Check my Running Processes on Login Node:"
                top -u $USER
                exit
        fi
        if [[ $REPLY == 4 ]]; then
                module load ams
                read -p "Enter your Project ID: " project_id
                project_summary $project_id
                exit
        fi
	if [[ $REPLY == 5 ]]; then
		clear
		check_license_abaqus.sh
		exit	
	fi
	if [[ $REPLY == 6 ]]; then
		clear
		check_license_matlab.sh
		exit
	fi
	if [[ $REPLY == 7 ]]; then 
		clear
		check_license_ansys.sh
		exit
	fi
	if [[ $REPLY == 8 ]]; then
		read -p "Enter your Project ID: " project_id
	 	countjobs=`qselect -P $project_id |wc -l`
		if [[ $countjobs -gt 0 ]]; then
			qstat -ans `qselect -P $project_id`
			exit
		else
			echo "No Running Projects"
		fi
		
	fi
else
        echo "Invalid entry." >&2
        exit 1
fi

Using BASH command Find to locate files – Part 2

Using User-Defined Actions:

P1. Prompt the user before findings.

[u1@node1 Downloads]$ find -type f -name 'rrr' -ok ls -l '{}' ';'
< ls ... ./rrr > ? Y
-rw-r--r-- 1 u1 users 0 Nov 19 10:36 ./rrr

where

  • {} is the symbolic representation of the current path
  • ; is the required delimiter indicating the end of the command
  • -ok can replace -exec so that the user can be prompted before executing any commands.

 

Pre-defined Action

P1. Find the file and delete the files immediately

[u1@node1 Downloads]$ find -type f -name '*.mov' -delete

P2. Find and quit once the match has been made

[u1@node1 Downloads]$ find -type f -name '*.mov' -quit

P3. Perform the equivalent of ls -dil on the match file.

[u1@node1 Downloads]$ find -type f -name '*.mov' -ls
4554823854 70266 -rw-r--r--   1 u1   users 62574861 Sep  5  2018 ./miniconda.sh

 

Using Operators

[u1@node1 Downloads]$ find \(-type -f -not -name '*.txt' \) -or \( -type -r -not -name '*.doc' \)
  • Parentheses have special meaning to the shell. Putting a backslash character in front of the parenthesis help to prevent shell from interpreting them.

 

Options

Set the maximum number of levels that find will descend when performing tests and actions.

[u1@hpc-gekko1 Downloads]$ find -maxdepth 2 -type f -name '*' -ls
4554823854 70266 -rw-r--r--   1 u1   users 62574861 Sep  5  2018 ./miniconda.sh

 

Using BASH command Find to locate files – Part 1

P1. Use command to list files in the lammps-7Aug19 directory

[u1@node1 lammps-7Aug19]$ find

Pt 2. Count the number of files by piping the list to wc to count the files.

[u1@node1 lammps-7Aug19]$ find | wc -l
14717

P3. Count the the directories listing by piping the list to wc to count the directories

[u1@node1 lammps-7Aug19]$ find -type d | wc -l
672

P4. Count the list of files that matched the wildcard pattern *.mov that are larger than 1M by piping to wc to count

[u1@node1 lammps-7Aug19]$ find -type f -name "*.jpg" -size +1M |wc -l

P5.  Find and match files or directories whose contents or attributes were last modified more recently than those of “mybasefile”

[u1@node1 lammps-7Aug19]$ find -type f -name "*.jpg" -cnewer mybasefile

P6. Find empty files in the lammps-7Aug19 directory

[u1@node1 lammps-7Aug19]$ find -type f -empty

Pt 7. Find files or directories that belongs to username

[u1@node1 Downloads]$ find -type f -user root |wc -l
1572

References:

  1. The Linux Command Line

Checking and Modifying Timestamp of whole Directory recursively

Step 1: Show the complete date, time and year for a specific file

$ ls -l --full-time
-rwxrwxr-x  1 root root  1109 2018-07-20 12:52:52.587945000 +0800 Allwmake
drwxrwxr-x  5 root root  4096 2018-07-20 12:52:52.602945000 +0800 applications
drwxrwxr-x  3 root root  8192 2018-07-20 12:53:19.536973000 +0800 bin
-rw-rw-r--  1 root root 35646 2018-07-20 12:52:52.592945000 +0800 COPYING
drwxrwxr-x  5 root root  4096 2018-07-20 12:53:19.936974000 +0800 doc
drwxrwxr-x  8 root root  4096 2018-07-20 12:53:20.039974000 +0800 etc
drwxr-xr-x  4 root root  4096 2018-07-20 12:55:17.230101000 +0800 platforms
-rw-rw-r--  1 root root  1620 2018-07-20 12:52:52.597945000 +0800 README.org
drwxrwxr-x 38 root root  4096 2018-07-20 12:53:22.032976000 +0800 src
drwxrwxr-x 17 root root  4096 2018-07-20 12:54:45.114064000 +0800 tutorials
drwxrwxr-x  7 root root  4096 2018-07-20 12:55:15.939099000 +0800 wmake

Step 2: If you wish to modify the time-stamp for the entire directory, you can use the command,

# for file in `find .`; do touch $file; done

References:

  1. touch – change file timestamps(Unix Tutorial)