Author: kittycool only

by

Installing Docker on CentOS 7

 

Point 1: Install the Dependencies

# yum install yum-utils device-mapper-persistent-data lvm2
  • yum-utils ->Include yum-config-manager
  • device-mapper-persistent-data and lvm2 -> Device Mapper Storage Drivers

Point 2: Add Docker Repository to CentOS

# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Point 3: Install Docker on CentOS

# yum install docker

Point 4: Manage Docker Service (Start, Enable and Check Status)

# systemctl start docker
# systemctl enable docker
# systemctl status docker 0

Point 5: Installing specific version of docker

# yum list docker-ce --showduplicates
docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable
.....
.....
.....
.....
.....
docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable

Point 5b:

# yum install docker-ce-18.06.3.ce

References:

  1. How to Install Docker on CentOS 7
by

Compile StringTie on CentOS 7.6

StringTie is a fast and highly efficient assembler of RNA-Seq alignments into potential transcripts.

System Information:

  • CentOS 7.6
  • GNU-4.8.5

The installation instruction can be found in the site

$ git clone https://github.com/gpertea/stringtie
$ cd stringtie
$ make release

Conduct Tests

$ ./run_tests.sh
..Downloading test data..
..unpacking test data..
Test 1: Short reads
OK.
Test 2: Short reads and super-reads
OK.
Test 3: Long reads
OK.
Test 4: Long reads with annotation guides
OK.
by

Compiling BoltzTraP-1.2.5 with GNU

BoltzmannTransportProperties (BoltzTraP) can be obtained from http://www.icams.de/boltztrap

Point 1: Got to boltztrap src directory

$ cd $HOME/boltztrap-1.2.5/src

Point 2: Edit the Makefile

# gfortran
SHELL = /bin/sh
FC = gfortran
LIBS = -llapack -lblas

# generic
LINKER  =       $(FC)
LFLAGS  = -g #-pg
FGEN =
DESTDIR = .

EXECNAME = BoltzTraP

###############################################################################
FFLAGS = $(FGEN) $(FOPT)
EXEC = $(DESTDIR)/$(EXECNAME)

#..............................................................................
#
#  Object files common to both REAL and COMPLEX type subroutines
#
OBJS = gmlib2.o reallocate.o \
     m_bandstructure.o m_input.o m_fermimod.o \
     m_interfaces.o \
     latgen2.o generic_field.o gtfnam.o gen_lattpoints.o \
     BoltzTraP.o crystal_band.o wien_band.o phon_band.o generic_band.o pw_interface.o \
     add_inv.o bandana.o stern1.o kdelta.o fite4.o sortag.o gplbands.o \
     dos.o ifflim.o setfft.o c3fft.o boseintegrals.o fermiintegrals.o bands.o kcomp.o \
     bz.o fermisurface.o setfft2.o write_dx_fs.o write_dx_bz.o write_cube_fs.o \
     dos_histogram.o dos_tetra.o noculc.o dosvv.o readvv.o \
     phonondrag.o
#OBJS = \
#        reallocate.o defs.o modules.o broad.o add_inv.o \
#        c3fft.o gtfnam.o ifflim.o mknam.o read_energy.o \
#        transport.o stern.o kdelta.o gen_lattpoints.o fite4.o setfft.o \
#        starfkt2.o dos.o

$(EXEC): $(OBJS)
        $(LINKER) $(LFLAGS) -o $(EXEC) $(OBJS) $(LDFLAGS) $(LIBS) $(LFLAGS)


clean:
        rm -f *.o *.mod *.pc *.pcl *~

.SUFFIXES: .F90 .o
.F90.o:
        $(FC) $(FFLAGS) -c $<

3. Make Install

$ make

You should see a BoltzTraP executable in the src directory

by

Unable to use “-v” variable in PBS Professional 19.2.5

I was not able to use the “-v file=test.m” in the latest version of PBS Professional 19.2.5

I was using the following commands and the qsub command did not work. It used to work in earlier version of PBS Professional

$ qsub gpu.pbs -v file=test.m
usage: qsub [-a date_time] [-A account_string] [-c interval]
[-C directive_prefix] [-e path] [-f ] [-h ] [-I [-X]] [-j oe|eo] [-J X-Y[:Z]]
[-k keep] [-l resource_list] [-m mail_options] [-M user_list]
[-N jobname] [-o path] [-p priority] [-P project] [-q queue] [-r y|n]
[-R o|e|oe] [-S path] [-u user_list] [-W otherattributes=value...]
[-S path] [-u user_list] [-W otherattributes=value...]
[-v variable_list] [-V ] [-z] [script | -- command [arg1 ...]]
qsub --version

The solution is that by design the job script has to be the last argument. Please change the commands accordingly.

$ qsub -v file=test.m gpu.pbs
by

Intel® Math Kernel Library Link Line Advisor

The Intel® Math Kernel Library (Intel® MKL) is designed to run on multiple processors and operating systems. It is also compatible with several compilers and third party libraries, and provides different interfaces to the functionality. To support these different environments, tools, and interfaces Intel MKL provides mutliple libraries from which to choose.

 

For more information to generate the libraries  Intel® Math Kernel Library Link Line Advisor

 

by

Configure PBS not to accept jobs that will run into Scheduled Down-Time

Step 1: Go to /pbs/pbs_home/sched_priv and edit the file dedicated_time

# vim /pbs/pbs_home/sched_priv/dedicated_time

Edit the Start and End Date Time in the given format

# FORMAT: FROM TO
# ---- --
# MM/DD/YYYY HH:MM MM/DD/YYYY HH:MM
For example

01/08/2020 08:00 01/08/2020 20:00

Step 2: Reload the pbs configuration by sending a SIGHUP

# ps -eaf | grep -i pbs_sched
# kill -HUP 438652

Step 3: Submit a job that cross over the scheduled date and time, you should see

$ qstat -asn1
55445.hpc-mn1 user1 q32 MPI2 -- 3 96 -- 120:0 Q -- --Not Running: Job would cross dedicated time boundary
55454.hpc-mn1 user2 q32 MPI -- 1 4 -- 120:0 Q -- --Not Running: Job would cross dedicated time boundary
55455.hpc-mn1 user1 q32 MPI -- 1 4 -- 120:0 Q -- --Not Running: Job would cross dedicated time boundary
.....
.....
by

Installing conda packages locally in own directory

Step 1: Module load the anaconda module in HPC (if you are using Module Environment)

$ module load anaconda2019/python3

Step 2: Create a virtual environment locally and install packages

$ conda create -y -n my-own-conda-env

You can rename my-own-conda-env with any name. It is good practise to have a “-env” postfix.

Step 2(Option) : Install conda-env in non-default directory

If you wish to install in other directory instead of the default ~/.conda/envs, you have to use the “-p” parameters

$ conda create -y -p ~/my-conda-direct-env

Step 3: Activate the conda environment

$ source activate my-own-conda-env

You should see the environment activated which is prefix the login prompt.

(conda-env) [user1@hpc-n001 ~]$

Step 4: Download and install the conda package

$ conda install -c  ipyrad ipyrad

Step 5: Check for installed package

$ conda list

Step 6: Deactivate environment

$ source deactivate

The command prompt will be revert back without the conda environment

[user1@hpc-n001 ~]$
by

Conda versus pip virtualenv commands

I find this comparison very useful when trying to understand conda and pip. The information has been taken from Conda Command Conference

 

Task Conda package and environment manager command Pip package manager command
Install a package conda install $PACKAGE_NAME pip install $PACKAGE_NAME
Update a package conda update --name $ENVIRONMENT_NAME $PACKAGE_NAME pip install --upgrade $PACKAGE_NAME
Update package manager conda update conda Linux/macOS: pip install -U pip Win: python -m pip install -U pip
Uninstall a package conda remove --name $ENVIRONMENT_NAME $PACKAGE_NAME pip uninstall $PACKAGE_NAME
Create an environment conda create --name $ENVIRONMENT_NAME python X
Activate an environment conda activate $ENVIRONMENT_NAME* X
Deactivate an environment conda deactivate X
Search available packages conda search $SEARCH_TERM pip search $SEARCH_TERM
Install package from specific source conda install --channel $URL $PACKAGE_NAME pip install --index-url $URL $PACKAGE_NAME
List installed packages conda list --name $ENVIRONMENT_NAME pip list
Create requirements file conda list --export pip freeze
List all environments conda info --envs X
Install other package manager conda install pip pip install conda
Install Python conda install python=x.x X
Update Python conda update python* X

References:

  1. Conda Command Conference
by

Understanding Conda and Pip

The information on understanding Conda and Pip by Anaconda is really good. The summary of the table is below

Comparison of conda and pip

conda pip
manages binaries wheel or source
can require compilers no yes
package types any Python-only
create environment yes, built-in no, requires virtualenv or venv
dependency checks yes no
package sources Anaconda repo and cloud PyPI

References: