Application

by

KIM_SimulatorHeaders.h: No such file or directory during compilation of lammps

If you encounter an error such as the one below during compilation for lammps such as written in Compiling LAMMPS-15Jun20 with GNU 6 and OpenMPI 3

After running the command, 

% make g++_openmpi -j 16

you may encounter

In file included from ../style_pair.h:112:0,
                 from ../force.cpp:20:
../pair_kim.h:70:34: fatal error: KIM_SimulatorHeaders.h: No such file or directory
 #include "KIM_SimulatorHeaders.h"

This is due to one of the a header file not found at /usr/local/lammps-29Oct20/src/pair_kim.h. At line 70, modify the actual location of KIM_SimulatorHeaders.h 

/* #include "KIM_SimulatorHeaders.h" */
#include "/usr/local/lammps-29Oct20/lib/kim/kim-api-2.1.3/build/installed-kim-api-2.1.3/include/kim-api/KIM_SimulatorHeaders.h"

Again, after running the command, 

% make g++_openmpi -j 16

you may encounter

mpicxx -g -O3  -DLAMMPS_GZIP -I../../lib/colvars -DLMP_USER_OMP -I../../lib/voronoi/includelink -DLMP_PYTHON -I../../lib/poems -DLMP_MPIIO -I../../lib/message/cslib/src -I../../lib/latte/includelink -DLMP_KOKKOS  -DMPICH_SKIP_MPICXX -DOMPI_SKIP_MPICXX=1     -I/usr/include/python2.7 -I/usr/include/python2.7   -I./ -I../../lib/kokkos/core/src -I../../lib/kokkos/containers/src -I../../lib/kokkos/algorithms/src --std=c++11 -fopenmp -I./ -I../../lib/kokkos/core/src -I../../lib/kokkos/containers/src -I../../lib/kokkos/algorithms/src   -DLMP_KIM_CURL   -c ../kim_param.cpp
../kim_param.cpp:73:34: fatal error: KIM_SimulatorHeaders.h: No such file or directory
 #include "KIM_SimulatorHeaders.h"

This is due to one of the a header file not found at /usr/local/lammps-29Oct20/src/kim_param.cpp. At line 73, modify the actual location of KIM_SimulatorHeaders.h 

/* #include "KIM_SimulatorHeaders.h" */
#include "/usr/local/lammps-29Oct20/lib/kim/kim-api-2.1.3/build/installed-kim-api-2.1.3/include/kim-api/KIM_SimulatorHeaders.h"

There might be the same issue for kim_init.cpp, fix_store_kim.cpp, pair.kim.h etc

by

Issues when launching ABAQUS/CAE

I was using FastX3, loaded the ABAQUS 2020 cae, 

% module load abaqus/2020
% abaqus cae
 Error: code 2 major 153 minor 3: BadValue (integer parameter out of range for operation).
X Error: code 167 major 153 minor 5: GLXBadContext.
X Error: code 167 major 153 minor 26: GLXBadContext.
X Error: code 167 major 153 minor 4: GLXBadContext.
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
failed to create drawable
.....
.....
.....



Warning: There was a problem creating an OpenGL feedback context for printing.
If you encounter any problems with printing, verify that indirect GLX context
creation is enabled on your X server. For more information on indirect / direct
GLX context and how to enable indirect GLX context creation, see Knowledge Base
article QA00000043316

The issue is that on some graphics devices Abaqus/CAE and Abaqus/Viewer may fail when hardware acceleration is turned on. To mitigate the issue, you can turned off hardware acceleration which may have some performance degradation for the graphics performance. This can be done by simply

% abaqus cae --mesa

ABAQUS maintained a list of Graphics Devices which has passed the test and you may want to take a look

ABAQUS 2020 Graphics Devices

by

Setting up FreeSurfer 7.2.0 for CentOS-7

FreeSurfer is an open source software suite for processing and analyzing (human) brain MRI images.

  • Skullstripping
  • Image Registration
  • Subcortical Segmentation
  • Cortical Surface Reconstruction
  • Cortical Segmentation
  • Cortical Thickness Estimation
  • Longitudinal Processing
  • fMRI Analysis
  • Tractography
  • FreeView Visualization GUI
  • and much more..

The freesurfer downloads can be found here

The Tar GZ for Linux Install can be found here 

% tar -zxvpf freesurfer-linux-centos7_x86_64-7.0.0.tar.gz
% cd freesurfer

Prepare the Environment in your .bashrc

export FREESURFER_HOME=/usr/local/freesurfer
export FSFAST_HOME=$FREESURFER_HOME/fsfast
export FSF_OUTPUT_FORMAT=nii.gz
export FMRI_ANALYSIS_DIR=$FREESURFER_HOME/fsfast
export FUNCTIONALS_DIR=$FREESURFER_HOME/sessions
export FS_OVERRIDE=0
export MNI_DIR=$FREESURFER_HOME/mni
export MINC_BIN_DIR=$FREESURFER_HOME/mni/bin
export MINC_LIB_DIR=$FREESURFER_HOME/mni/lib
export MNI_DATAPATH=$FREESURFER_HOME/mni/data
export PERL5LIB=$FREESURFER_HOME/mni/share/perl5
export MNI_PERL5LIB=$FREESURFER_HOME/mni/share/perl5

by

Compiling dcm2niix on CentOS 7

Prerequisites

  1. GNU 6.5
  2. > CMAKE 3.9.6

What is dcm2niix?

dcm2niix is designed to convert neuroimaging data from the DICOM format to the NIfTI format. This web page hosts the developmental source code – a compiled version for Linux, MacOS, and Windows of the most recent stable release is included with MRIcroGL. A full manual for this software is available in the form of a NITRC wiki.

Compilation 

% git clone https://github.com/rordenlab/dcm2niix.git
% cd dcm2niix
% cmake -DUSE_OPENJPEG=ON -DCMAKE_CXX_FLAGS=-g -DUSE_STATIC_RUNTIME:BOOL=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/dcm2niix . && make
% make install

If you turn USE_STATIC_RUNTIME:BOOL=ON, Some Centos/Redhat may report “/usr/bin/ld: cannot find -lstdc++”. This can be resolved by installing static versions of libstdc++: yum install libstdc++-static.

by

voro++.hh: No such file or directory during compilation of lammps with voronoi

If you encounter an error such as the one below during compilation for lammps such as written in Compiling LAMMPS-15Jun20 with GNU 6 and OpenMPI 3 

../compute_voronoi_atom.h:24:21: fatal error: voro++.hh: No such file or directory

This is due to one of the a header file not found at /usr/local/lammps-29Oct20/src/compute_voronoi_atom.cpp. To resolve the issue, do take a look at Line 23 or 24 and edit to the path where you place voro++.hh

/* #include <voro++.hh> */
#include "/usr/local/lammps-29Oct20/lib/voronoi/voro++-0.4.6/src/voro++.hh"

References:

  1. https://sourceforge.net/p/lammps/mailman/message/31832584/
by

Resolving GNU MP not found on CentOS 7

If you are installing package like BiocManager::install(“scDblFinder”)

BiocManager::install("scDblFinder")

You may encounters error like 

configure: error: GNU MP not found, or not 4.1.4 or up, see http://gmplib.org

The fix is obvious as seen on error message. If you are using CentOS 7, you can easily fix it via Yum

% yum install gmp-devel
by

Compiling R-4.1.0 with GNU

The R Project for Statistical Computing

Prerequisites

gnu-6.5
m4-1.4.18
gmp-6.1.0
mpfr-3.1.4
mpc-1.0.3
isl-0.18
gsl-2.1

Compiling PCRE is important or you will face an error like

configure: error: PCRE2 library and headers are required, or use --with-pcre1 and PCRE >= 8.32 with UTF-8 support

After you have compile PCRE, you can proceed with the compilation of R-4.1.0

% ./configure --prefix=/usr/local/R-4.1.0 --with-pcre1=/usr/local/pcre-8.42 --with-blas --with-lapack --enable-R-shlib

If there are no issues…. 

R is now configured for x86_64-pc-linux-gnu

  Source directory:            .
  Installation directory:      /usr/local/R-4.1.0

  C compiler:                  gcc  -g -O2
  Fortran fixed-form compiler: gfortran  -g -O2

  Default C++ compiler:        g++ -std=gnu++14  -g -O2
  C++11 compiler:              g++ -std=gnu++11  -g -O2
  C++14 compiler:              g++ -std=gnu++14  -g -O2
  C++17 compiler:
  C++20 compiler:
  Fortran free-form compiler:  gfortran  -g -O2
  Obj-C compiler:              gcc -g -O2 -fobjc-exceptions

  Interfaces supported:        X11
  External libraries:          pcre1, readline, BLAS(generic), LAPACK(generic), curl
  Additional capabilities:     PNG, JPEG, NLS, ICU
  Options enabled:             shared R library, R profiling

  Capabilities skipped:        TIFF, cairo
  Options not enabled:         shared BLAS, memory profiling

  Recommended packages:        yes

Make and Make Install the Files

% make -j 8
.....
gcc -I"/home/user1/Downloads/R-4.1.0/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c anova.c -o anova.o
gcc -I"/home/user1/Downloads/R-4.1.0/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c anovapred.c -o anovapred.o
gcc -I"/home/user1/Downloads/R-4.1.0/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c branch.c -o branch.o
gcc -I"/home/user1/Downloads/R-4.1.0/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c bsplit.c -o bsplit.o
gcc -I"/home/user1/Downloads/R-4.1.0/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2  -c choose_surg.c -o choose_surg.o
g
.....
% make install

References:

  1. Compiling R by Toby Dylan
by

Basic CURL Commands

curl is a command line tool to transfer data to or from a server. It is able to use any of the supported protocols like HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TFTP, TELNET, LDAP or FILE. This tool is very useful for automation, since it is designed to work without user interaction. Furthermore, curl can transfer multiple file at once.

Basic Single URL Usage

% curl https://thelinuxcluster.com/

2a. Save the Download File with a preferred file name

Save the Download File on the local machine with the name provided with the parameter.

% curl -o test.o https://thelinuxcluster.com/test.output

2b. Save the Download File

% curl -O https://thelinuxcluster.com/test.output

2c. Download Multiple Files. Just Multiple -O

% curl -O https://thelinuxcluster.com/CentOS1.iso -O https://thelinuxcluster.com/CentOS2.iso -O https://thelinuxcluster.com/CentOS3.iso

3a. Display a Progress Meter

% curl --progress-bar -o test.o https://thelinuxcluster.com/test.output

3b. Do not display a Progressive Bar

% curl --silent -o test.o https://thelinuxcluster.com/test.output

4 Limit Rate of Data Transfer

% curl --limit-rate 1000K -o test.o https://thelinuxcluster.com/CentOS1.iso

5a Uploading a File to the FTP Server

% curl -u username:userpassword -T myfile ftp://ftp.thelinuxcluster.com/

5b. Appending the File to the FTP Server

% curl -u username:userpassword -a -T myfile ftp://ftp.thelinuxcluster.com/

5c Downloading the File to the File Server

% curl ftp:/ftp.thelinuxcluster.com/CentOS79.iso --user username:userpassword -o myCentOS79.iso

6a. Verifying SSL Certificate

% curl --cacert server.crt https://thesupersecureserver.com

6b. Ignoring SSL Certificate

% curl -k https://thesupersecureserver.com/

7a Proxy Server

% curl -x proxy_name:proxy_port https://thelinuxcluster.com

7b Proxy Server which requires authentication

% curl --user username:userpassword -x proxy_name:proxy_port https://thelinuxcluster.com

8 Sending Email

% curl --url "smtps"//smtp.thelinuxcluster.com:465: --ssl-reqd --mail-from "sender@thelinuxcluster.com" --mail-rcpt "receiver@thelinuxcluster.com" --upload-file maincontent.txt --user "sender@thelinuxcluster.com:password" --insecure

References:

  1. Learn to use CURL command with examples
  2. Curl command in Linux with Examples

by

Creating Virtual Environment with Python using venv

Virtual Environment and Packages

Sometimes there is a need to create virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages as there is sometimes a need for specific version of python library. Here come venv

Creating Virtual Environments 

% module load python/3/intel/2020
% python3 -m venv myenv

This will create a myenv directory if it doesn’t exist, and also create directories inside it containing a copy of the Python interpreter, the standard library, and various supporting files. 

% source myenv/bin/activate

Activating the virtual environment will change your shell’s prompt to show what virtual environment you’re using,

(tutorial-env) [user1@hpc-node1 bin]$ python
Python 3.7.4 (default, Nov 22 2019, 21:31:39)
[GCC 7.3.0] :: Intel(R) Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distibution
>>> import sys
>>> sys.path
['', '/usr/local/intelpython3/lib/python37.zip', ....., '.....', '/myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages']
>>>

Managing Packages with pip

You can install, upgrade and remove using a program called pip. By default, pip will install packages from the Python Package Index (http://pypi.org).

You can install the latest version of a package by specifying a package’s name:

(tutorial-env) [user1@hpc-node1 bin]$ python -m pip install novas
Collecting novas
  Downloading https://files.pythonhosted.org/packages/42/95/a05bc35cb119925e10f9faa8a2bd17020b0a585744a38921a709acdd9a14/novas-3.1.1.5.tar.gz (135kB)
    100% |████████████████████████████████| 143kB 4.6MB/s
Installing collected packages: novas
  Running setup.py install for novas ... done
Successfully installed novas-3.1.1.5

You can also install specific version by using the command 

(tutorial-env) [user1@hpc-node1 bin]$ python -m pip install requests==2.6.0
Collecting requests==2.6.0
  Downloading https://files.pythonhosted.org/packages/73/63/b0729be549494a3e31316437053bc4e0a8bb71a07a6ee6059434b8f1cd5f/requests-2.6.0-py2.py3-none-any.whl (469kB)
    100% |████████████████████████████████| 471kB 4.3MB/s
Installing collected packages: requests
Successfully installed requests-2.6.0

You can also upgrade the package by using the command

(tutorial-env) [user1@hpc-node1 bin]$ python -m pip install --upgrade request    s
Collecting requests
  Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50    280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl     (61kB)
    100% |████████████████████████████████| 61kB 3.4MB/s
Collecting chardet<5,>=3.0.2 (from requests)
  Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87    d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl (1    78kB)
    100% |████████████████████████████████| 184kB 5.3MB/s
Collecting urllib3<1.27,>=1.21.1 (from requests)
  Downloading https://files.pythonhosted.org/packages/0c/cd/1e2ec680ec7b09846dc    6e605f5a7709dfb9d7128e51a026e7154e18a234e/urllib3-1.26.5-py2.py3-none-any.whl (    138kB)
    100% |████████████████████████████████| 143kB 5.9MB/s
Collecting idna<3,>=2.5 (from requests)
  Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6    f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 4.3MB/s
Collecting certifi>=2017.4.17 (from requests)
  Downloading https://files.pythonhosted.org/packages/05/1b/0a0dece0e8aa492a6ec    9e4ad2fe366b511558cdc73fd3abc82ba7348e875/certifi-2021.5.30-py2.py3-none-any.wh    l (145kB)
    100% |████████████████████████████████| 153kB 5.1MB/s
Installing collected packages: chardet, urllib3, idna, certifi, requests
  Found existing installation: requests 2.6.0
    Uninstalling requests-2.6.0:
      Successfully uninstalled requests-2.6.0
Successfully installed certifi-2021.5.30 chardet-4.0.0 idna-2.10 requests-2.25.    1 urllib3-1.26.5

You can display all the packages in the environment in virtual environment

(tutorial-env) [user1@hpc-node1 bin]$ pip list
Package    Version
---------- ---------
certifi    2021.5.30
chardet    4.0.0
idna       2.10
novas      3.1.1.5
pip        19.0.3
requests   2.25.1
setuptools 40.8.0
urllib3    1.26.5

pip freeze can be used to produce a similar list of installed packages and formatted in a output file that pip install can read from

(tutorial-env) [user1@hpc-node1 bin]$ pip freeze > requirements.txt
(tutorial-env) [user1@hpc-node1 bin]$ cat requirements.txt
certifi==2021.5.30
chardet==4.0.0
idna==2.10
novas==3.1.1.5
requests==2.25.1
urllib3==1.26.5

Install all the necessary packages with install -r from requirements.txt

(tutorial-env) [user1@hpc-node1 bin]$ python -m pip install -r requirements.txt
Requirement already satisfied: certifi==2021.5.30 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (2021.5.30)
Requirement already satisfied: chardet==4.0.0 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (4.0.0)
Requirement already satisfied: idna==2.10 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (2.10)
Requirement already satisfied: novas==3.1.1.5 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (3.1.1.5)
Requirement already satisfied: requests==2.25.1 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (2.25.1)
Requirement already satisfied: urllib3==1.26.5 in /myhome/melvin/python_project/tutorial-env/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (1.26.5)

References:

by

RELION – Performance Benchmark and Profiling

What is RELION?

RELION (REgularized LIkelihood OptimizatioN) is an open-source program for the refinement of macromolecular structures by single-particle analysis of electron cryomicroscopy (cryo-EM) data

RELION (REgularized LIkelihood OptimizatioN) implements an empirical Bayesian approach for analysis of electron cryo-microscopy (Cryo-EM)

RELION provides refinement methods of singular or multiple 3D reconstructions as well as 2D class averages

RELION is an important tool in the study of living cells

HPC-AI Advisory Council

Performance Analysis Summary

(from Article See RELION – Performance Benchmark and Profiling)

RELION performance testing

  • Pool size 4,8,16 gave best performance on 16,24,32 nodes
  • SHARP In-Network Computing reduces MPI time by 13% and increase overall application performance by 5
  • Performance advantages increases with system size, up to 32 nodes were tested

RELION Profile

  • Rank #0 does not perform computation
  • Mostly MPI_Barrier (70%)
  • Ring communication matrix

References: