Author: kittycool only

by

Using vmstat for profiling

Taken from RHEL Performance Tuning Student Booklet

The vmstat command, if given no arguments, will print out the averages of various system statistics since boot. The vmstat command accepts two arguments. The first is the delay, and the second is the count. The delay is a value in seconds between output. The count is the number of iterations of statistics to report.

[root@lime ~]# vmstat 4 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
0  0   2832 145216 194528 6772048    0    0     0     1    0    1  0  0 100  0  0
0  0   2832 145200 194528 6772048    0    0     0     1   44   45  0  0 100  0  0
0  0   2832 145208 194528 6772048    0    0     0     8   46   45  0  0 100  0  0
1  0   2832 145208 194528 6772048    0    0     0     0   44   46  0  0 100  0  0
0  0   2832 145208 194528 6772048    0    0     0     3   46   55  0  0 100  0  0
Category Statistics Definition
Process related r The number of processes waiting for runtime
Process related b The number of processes in uninterruptible sleep
memory swpd The amount of memory currently used in swap
spaces
memory free The amount of idle (immediately available)
memory
memory buff The amount of memory used as buffers
memory cache The amount of memory used as cache
swap: paging statistics si Pages of memory swapped in per second
swap: paging statistics so Pages of memory swapped out per second
io: block I/O statistics bi Blocks per second received from block devices
io: block I/O statistics bo Blocks per second sent to block devices
system in Interrupts raised per second
system cs Context switches per second
cpu: how CPU time is used us Percentage of time spent running user-space code
cpu: how CPU time is used sy Percentage of time spent running kernel code
cpu: how CPU time is used id Percentage of time spent idle
cpu: how CPU time is used wa Percentage of time spent blocked while waiting for
I/O to complete
cpu: how CPU time is used st Percentage of time where the CPU had a process
ready to run, but CPU time was stolen by the
hypervisor supporting this virtual machine
(typically because the CPU is being used by
another guest virtual machine)

 

by

Determine how many CPUs in the systems

I came across this sniffy forgotten tools to help determine the CPU information. I guess we are all familiar with top. But if you type “1” after top, you are able to see details in the individual

A. Using top

# top

top - 10:11:35 up 95 days, 20:02,  2 users,  load average: 0.51, 0.39, 0.20
Tasks: 326 total,   1 running, 325 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.1%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65951272k total, 65647784k used,   303488k free,   221504k buffers
Swap: 33046520k total,        0k used, 33046520k free, 63608900k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13469 root      20   0 15172 1436  956 R  2.6  0.0   0:00.72 top
1 root      20   0 19352 1520 1208 S  0.0  0.0   0:01.58 init
2 root      20   0     0    0    0 S  0.0  0.0   0:00.01 kthreadd
..............
..............
1

top - 10:11:58 up 95 days, 20:03,  2 users,  load average: 0.46, 0.38, 0.21
Tasks: 310 total,   1 running, 309 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.3%sy,  0.0%ni, 99.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
.........

B. Using lspcu to take a look at summary

# lscpu

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                12
On-line CPU(s) list:   0-11
Thread(s) per core:    1
Core(s) per socket:    6
CPU socket(s):         2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 45
Stepping:              7
CPU MHz:               2501.000
BogoMIPS:              4999.27
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              15360K
NUMA node0 CPU(s):     0-5
NUMA node1 CPU(s):     6-11

 

by

Commands for sending signals by explicit request

A. Foreground Processes:

You can use the keyboard to issue a signal on your current processing by pressing a keyboard control sequence t

1. Suspend foreground process

# Ctrl+z

2. Kill foreground process

# Ctrl+c

3. Core Dump

# Ctrl+\

B. Background Process

  1. Check the list of kill option
# kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
..................................
# kill -9 PID

2. Look up processes based on name and kill them

# pgrep -l -u user1
7000 bash
7001 sleep
...............
# pkill -SIGKILL -u user1

Notes: SIGTERM is the default signal, SIGKILL is a commonly misused administrator favorite. Since the SIGKILL signal cannot be handled or ignored, it is always fatal. However, it forces termination without allowing the killed process to run self-cleanup routines. It is recommended to send SIGTERM first, then retry with SIGKILL only if a process fails to respond

3. kill processes running in tty3

# pkill -SIGKILL -t tty3

4. Use the pstree command to view a process tree for the system or a single user.

# pstree -p root

[root@lime ~]# pstree -p root
init(1)─┬─NetworkManager(1785)
├─abrtd(2232)
by

Introduction to Systemd on CentOS 7

A few terms we need to grasp:

  1. Daemons are processes that wait or run in the background performing various tasks.
  2. To listen for connections, a daemon uses a socket.
  3. service often refers to one or more daemons

If you are moving from CentOS 6 to CentOS 7, you may be wondering why the need to move to systemd. Here are the features

  1. Parallelization capabilities, which increase the boot speed of a system
  2. Automatic service dependency management which can prevent long time-out.
  3. A method of tracking related processes together by using Linux control groups
  4. On-demand starting of daemons without requiring a separate service

Listing unit files with systemctl

  1. Query the state of all units 
    # systemctl
  2. Query the state of selected service 
    # systemctl --type=service
  3. List full output of the selected service. Useful for detailed check and investigation 
    # systemctl status sshd.service -l
  4. To check whether the particular is active and enabled to start at boot time 
    # systemctl is-active sshd
# systemctl is-enabled sshd
  5. List the active state of all loaded units. –all will include inactive units 
    # systemctl list-units --type=service
# systemctl list-units --type=service --all
  6. View the enabled and disabled settings for all units 
    # systemctl list-unit-files --type=service
  7. View only failed services. 
    # systemctl --failed --type=service

     

Controlling System Services

a. Status of a Service

# systemctl status sshd.service

b. Disable of a Service

# systemctl disable sshd.service

c. Enable and verify the status of a Service

# systemctl enable sshd.service
# systemctl is-enabled sshd.service

d. Reload configuration file of a running service

# systemctl is-enabled sshd.service
by

Input/Output Error at General Parallel File System (GPFS)

I was having this error “Input/output error on /usr/local/………….”

I did a mmdiag –waiters on all the NSD Nodes. There are no issues reported.

# mmlsdisk gpfs0
disk         driver   sector failure holds    holds                            storage
name         type       size   group metadata data  status        availability pool
------------ -------- ------ ------- -------- ----- ------------- ------------ ------------
dcs3700A_2   nsd         512    4714 yes      yes   ready         down         system
dcs3700A_3   nsd         512    4715 yes      yes   ready         up           system
dcs3700A_4   nsd         512    4714 yes      yes   ready         down         system
dcs3700A_5   nsd         512    4715 yes      yes   ready         up           system
dcs3700A_6   nsd         512    4714 yes      yes   ready         up           system
dcs3700A_7   nsd         512    4715 yes      yes   ready         up           system
dcs3700B_2   nsd         512    4716 yes      yes   ready         up           system
dcs3700B_3   nsd         512    4717 yes      yes   ready         up           system
dcs3700B_4   nsd         512    4716 yes      yes   ready         up           system
dcs3700B_5   nsd         512    4717 yes      yes   ready         up           system
dcs3700B_6   nsd         512    4716 yes      yes   ready         up           system
dcs3700B_7   nsd         512    4717 yes      yes   ready         up           system
[root@gpfsnsd1]# mmchdisk gpfs0 start -d dcs3700A_2
mmnsddiscover:  Attempting to rediscover the disks.  This may take a while ...
mmnsddiscover:  Finished.
gpfsnsd6:  Rediscovered nsd server access to dcs3700A_2.
gpfsnsd5:  Rediscovered nsd server access to dcs3700A_2.
gpfsnsd10:  Rediscovered nsd server access to dcs3700A_2.
gpfsnsd9:  Rediscovered nsd server access to dcs3700A_2.
Scanning file system metadata, phase 1 ...
6 % complete on Wed Apr 27 13:33:47 2016
25 % complete on Wed Apr 27 13:33:50 2016
41 % complete on Wed Apr 27 13:33:53 2016
58 % complete on Wed Apr 27 13:33:56 2016
76 % complete on Wed Apr 27 13:33:59 2016
90 % complete on Wed Apr 27 13:34:02 2016
100 % complete on Wed Apr 27 13:34:04 2016
Scan completed successfully.
Scanning file system metadata, phase 2 ...
Scan completed successfully.
Scanning file system metadata, phase 3 ...
Scan completed successfully.
Scanning file system metadata, phase 4 ...
Scan completed successfully.
Scanning user file metadata ...
50.23 % complete on Wed Apr 27 13:34:25 2016  (  26607616 inodes   38050968 MB)
100.00 % complete on Wed Apr 27 13:34:45 2016
Scan completed successfully.

References:

  1. Displaying GPFS™ disk states
  2. Changing GPFS disk states and parameters
by

Compiling and Installing mfix-2016.1

The document to install MFIX can be found at https://mfix.netl.doe.gov/download/mfix/mfix_current_documentation/mfix_user_guide.pdf

We compiled using Intel 12.1.5 and OpenMPI-1.6.5. Once done, you can easily compile with the following configuration parameters

$ ./configure_mfix FC=mpif90 FCFLAGS='-g -O2' --prefix=/usr/local/mfix-2016.1 --dmp
# make -j 16

Copy the libmfix.a to /usr/local/mfix-2016.1

# mkdir /usr/local/mfix-2016.1/lib
# cp libmfix.a /usr/local/mfix-2016.1/lib
by

Compiling LIGGGHTS with OpenMPI

LIGGGHTS stands for LAMMPS improved for general granular and granular heat transfer simulations.

There are similarities compiling LAMMPS and LIGGGHTS since both applications are related. Do take a look at Compiling LAMMPS-7Dec2015 with OpenMPI and GNU

FFTW, OpenMPI and GNU compilation are not covered here.

Step1: Download Download LIGGGHTS®-PUBLIC Using GIT. For more information see http://www.cfdem.com/system/files/githubaccess_public_1.pdf 

# git clone https://github.com/CFDEMproject/LIGGGHTS-PUBLIC.git
# git clone https://github.com/CFDEMproject/CFDEMcoupling-PUBLIC.git

to get the latest version, open a terminal,

# git pull

Step 2: Configure and Compile LIGGGHTS

# cd LIGGGHTS-PUBLIC/src/MAKE
# vim Makefile.openmpi

In Line 41,42,43 AND  52,53,54 comments the line or leave it blank

#MPI_INC =
#MPI_PATH =
#MPI_LIB =

#FFT_INC =
#FFT_PATH =
#FFT_LIB =

Step 3: Check and Build Packages
Check and Build Standard Package and exclude packages you will not need.

# make package-status
Installed  NO: package COHESIONMODELS
Installed YES: package MOLECULE
Installed YES: package PASCAL
Installed YES: package POEMS
Installed YES: package RIGID
Package.sh: line 7: cd: SURFACEMODELS: No such file or directory
Installed  NO: package SURFACEMODELS
Installed YES: package VORONOI

For example, exclude those you do not need.

# make yes-standard
# make no-asphere
# make no-voronoi
# make no-pascal
# make no-poems

Step 4: Compile LIGGGHTS
Check make options. Go to src

# make openmpi
# make lmp_openmpi

Step 5: Create a /usr/local/liggghts-12Jan16 and copy libraries.

# cp -Rv doc /usr/local/liggghts-12Jan16
# cp -Rv examples /usr/local/liggghts-12Jan16
# cp -Rv python /usr/local/liggghts-12Jan16
# cp README /usr/local/liggghts-12Jan16
# cp -Rv lib /usr/local/liggghts-12Jan16 
# cp lmp_openmpi /usr/local/liggghts-12Jan16/bin

Create a softlinks

# ln -s lmp_openmpi lggghts
by

Compiling LAMMPS-7Dec2015 with OpenMPI and GNU

From what I know for LAMMPS-7Dec2015, you should be able to e FFTW3. The current version at this point in writing is FFTW-3.3.4 (http://www.fftw.org/download.html)

Step 1: Compiling FFTW-3.3.4

# tar -zxvf fftw-3.3.4.tar.gz
# cd fftw-3.3.4
# ./configure --prefix=/usr/local/fftw-3.3.4 CC=mpicc F77=mpif90
# make -j 16
# make install

Step 1a. Compile OpenMPI and GNU (Not covered here)

Step 2: Untar LAMMPS

# tar -zxvf lammps-stable.tar.gz

Step 3: Prepare libraries for lammps

3a. lib/reax

# make -f Makefile.gfortran

3b. lib/meam

# make -f Makefile.gfortran

3c. lib/poems

# make -f Makefile.g++

Step 4: Compile required the packages. Go to src directory

# cd src

4a. Check which packages are included

# make package-status
Installed YES: package ASPHERE
Installed YES: package BODY
Installed YES: package CLASS2
Installed YES: package COLLOID
Installed YES: package COMPRESS
Installed YES: package CORESHELL
Installed YES: package DIPOLE
Installed YES: package FLD
Installed  NO: package GPU
Installed YES: package GRANULAR
Installed  NO: package KIM
Installed YES: package KOKKOS
  src/pair_lj_sdk_kokkos.cpp does not exist
  src/pair_lj_sdk_kokkos.h does not exist
Installed YES: package KSPACE
Installed YES: package MANYBODY
Installed YES: package MC
Installed YES: package MEAM
Installed YES: package MISC
Installed YES: package MOLECULE
Installed YES: package MPIIO
Installed YES: package OPT
Installed YES: package PERI
Installed YES: package POEMS
Installed YES: package PYTHON
Installed YES: package QEQ
Installed YES: package REAX
Installed YES: package REPLICA
Installed YES: package RIGID
Installed YES: package SHOCK
Installed YES: package SNAP
Installed YES: package SRD
Installed  NO: package VORONOI
Installed YES: package XTC
.........

4b. Choose all the standard

# make yes-standard

4c. Exclude packages that are not required

# make no-voronoi
# make no-kim
# make no-gpu

Step 5: Edit src/MAKE/Makefile.mpi

Around Line 43, 44
Leave MPI_PATH and MPI_LIB empty.

MPI_PATH =
MPI_LIB =

Around Line 53,54,55

FFT_INC =  -I/usr/local/fftw-3.3.3-single/include -DFFT_FFTW
FFT_PATH = -L/usr/local/fftw-3.3.3-single/lib
FFT_LIB = -lfftw

Step 6: Compile LAMMPS

Check make options. Go to src

# make
# make ompi_g++

Step 7: Create a /usr/local/lammps-7Dec15 and copy libraries. Go to lammps root directory

# cp -Rv bench /usr/local/lammps-7Dec15
# cp -Rv doc /usr/local/lammps-7Dec15
# cp -Rv examples /usr/local/lammps-7Dec15
# cp -Rv potentials /usr/local/lammps-7Dec15
# cp README /usr/local/lammps-7Dec15
# cp -Rv tools /usr/local/lammps-7Dec15
# cp -Rv lib /usr/local/lammps-7Dec15/lib
# cp lmp_ompi_g++ /usr/local/lammps-7Dec15/bin

7a. Create a softlinks

# lns -s /usr/local/lammps-7Dec15/bin/lmp_ompi_g++ lammps

References:

  1. Quick install guide for LAMMPS on Linux cluster
by

Installing Berkeley UPC on CentOS 6 and GNU-4.8

Preparation

  1. Download the Berkeley UPC from the Download Site. I use the http://upc.lbl.gov/download/
  2. Compile GNU 4.8.x. Do take a look at Compiling GNU 4.8.1 on CentOS 6
  3. Compile OpenMPI on GNU-4.8.1

Compilation

Step 1: Make sure your PATH and LD_LIBRARY_PATH reflect the GNU and OpenMPI Path

Step 2: Untar and Compile Berkeley UPC

# tar -zxvf berkeley_upc-2.22.0.tar.gz
# cd berkeley_upc-2.22.0
# mkdir build
# cd build
# ../configure CC=gcc CXX=g++ MPI_CC=mpicc --prefix=/usr/local/berkeley_upc-2.22.0
# gmake

Step 3: Untar and Compile the local UPC-C translator

Download the Berkeley Translator from the Download site. http://upc.lbl.gov/download/source.shtml#translator

# tar -zxvf berkeley_upc_translator-2.18.0.tar.gz
# cd berkeley_upc_translator-2.18.0
# make

After completion of successful build, it will show you the translator location path. Copy the line

translator = /usr/local/berkeley_upc_translator-2.18.0/open64/osprey1.0/build_ia64

and replace the line at the following files

/usr/local/RH6_apps/berkeley_upc-2.22.0/dbg/etc/upcc.conf
/usr/local/RH6_apps/berkeley_upc-2.22.0/opt/etc/upcc.conf

Step 4: Finally make install Berkeley UPC

# make install

Step 5: Test

# upcc --version
# upcrun

References:

  1. Installing Berkeley UPC on a Linux cluster