Application

by

Conda versus pip virtualenv commands

I find this comparison very useful when trying to understand conda and pip. The information has been taken from Conda Command Conference

 

Task Conda package and environment manager command Pip package manager command
Install a package conda install $PACKAGE_NAME pip install $PACKAGE_NAME
Update a package conda update --name $ENVIRONMENT_NAME $PACKAGE_NAME pip install --upgrade $PACKAGE_NAME
Update package manager conda update conda Linux/macOS: pip install -U pip Win: python -m pip install -U pip
Uninstall a package conda remove --name $ENVIRONMENT_NAME $PACKAGE_NAME pip uninstall $PACKAGE_NAME
Create an environment conda create --name $ENVIRONMENT_NAME python X
Activate an environment conda activate $ENVIRONMENT_NAME* X
Deactivate an environment conda deactivate X
Search available packages conda search $SEARCH_TERM pip search $SEARCH_TERM
Install package from specific source conda install --channel $URL $PACKAGE_NAME pip install --index-url $URL $PACKAGE_NAME
List installed packages conda list --name $ENVIRONMENT_NAME pip list
Create requirements file conda list --export pip freeze
List all environments conda info --envs X
Install other package manager conda install pip pip install conda
Install Python conda install python=x.x X
Update Python conda update python* X

References:

  1. Conda Command Conference
by

Understanding Conda and Pip

The information on understanding Conda and Pip by Anaconda is really good. The summary of the table is below

Comparison of conda and pip

conda pip
manages binaries wheel or source
can require compilers no yes
package types any Python-only
create environment yes, built-in no, requires virtualenv or venv
dependency checks yes no
package sources Anaconda repo and cloud PyPI

References:

by

Installing rgdal_1.4-8.tar.gz with R-3.6.0 on CentOS 7

I tried installing rgdal_1.4.8.tar.gz with R-3.6.0  and there were a number of issues. Even though I have followed and installed according to “Compile gdal-2.4.3 on CentOS 7” I have the follow error.

Issue 1: Error: Proj.h not found in standard or given locations

.....
.....
checking C++11 support for GDAL >= 2.3.0... yes
checking GDAL version >= 1.11.4... yes
checking GDAL version <= 2.5.0... yes
checking gdal: linking with --libs only... yes
checking GDAL: gdal-config data directory readable... yes
checking GDAL: /usr/local/gdal-2.4.3/share/gdal/pcs.csv readable... yes
checking proj_api.h presence and usability... no
configure: error: proj_api.h not found in standard or given locations.
ERROR: configuration failed for package ‘rgdal’
.....

Apparently R is not able to find inside pkconfig for proj.pc file. To resolve it, do the following to your ~.bashrc environment.

export PKG_CONFIG_PATH=/usr/local/proj-5.2.0/lib/pkgconfig

You can verify by using the command

# pkg-config proj --libs
-L/usr/local/proj-5.2.0/lib -lproj

You can get more information via the thread proj_api.h not found in standard or given locations #1092

Issue 2: Proj_LIB Path Issues

.....
checking proj_api.h presence and usability... yes
checking PROJ version >= 4.8.0... yes
checking projects.h presence and usability... yes
checking PROJ.4: epsg found and readable... no
Error: proj/epsg not found
Either install missing proj support files, for example
the proj-nad and proj-epsg RPMs on systems using RPMs,
or if installed but not autodetected, set PROJ_LIB to the
correct path, and if need be use the --with-proj-share=
configure argument.
ERROR: configuration failed for package ‘rgdal’
.....

To resolve the PROJ_LIB Issue

export PROJ_LIB=/usr/local/proj-5.2.0/share/proj

You can more information from https://github.com/r-spatial/lwgeom/issues/28

Finally, you should be able to compile rgdal_1.4-8.tar.gz with R-3.6.0

# R CMD INSTALL rgdal_1.4-8.tar.gz
.....
** R
** data
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rgdal)
.....

Final Step: Testing and Verification

Load both gdal and proj libaries. Make sure you have compile gdal and proj libraries using the tutorial Compile gdal-2.4.3 on CentOS 7

$ module load proj/5.2.0
$ module load R/3.6.0
$ module load gdal/2.4.3
$ R
library(rgdal)
Loading required package: sp
rgdal: version: 1.4-8, (SVN revision 845)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 2.4.3, released 2019/10/28
Path to GDAL shared files: /usr/local/gdal-2.4.3/share/gdal
GDAL binary built with GEOS: FALSE
Loaded PROJ.4 runtime: Rel. 5.2.0, September 15th, 2018, [PJ_VERSION: 520]
Path to PROJ.4 shared files: /usr/local/proj-5.2.0/share/proj
Linking to sp version: 1.3-2

 

by

Compile gdal-2.4.3 on CentOS 7

Step 1: Make sure you have the following packages on CentOS 7

yum install gcc-c++
yum install gcc
yum install libpng
yum install libtiff

Step 2: Compile and Install Proj-5.2.0
a. Download Source from http://download.osgeo.org/proj/

# wget https://download.osgeo.org/proj/proj-5.2.0.tar.gz
# tar -zvxf proj-5.2.0.tar.gz
# cd proj-5.2.0
# ./configure --prefix=/usr/local/proj-5.2.0
# make -j 16
# make install

Step 3: Compile and Install gdal-2.4.3

# wget http://download.osgeo.org/gdal/2.4.3/gdal-2.4.3.tar.gz
# tar -zxvf gdal-2.4.3.tar.gz
# cd gdal-2.4.3
# ./configure --prefix=/usr/local/gdal-2.4.3 --with-proj=/usr/local/proj-5.2.0 LDFLAGS="-L/usr/local/proj-5.2.0/lib" 
CPPFLAGS="-I/usr/local/proj-5.2.0/include" --with-threads --with-libtiff=internal --with-geotiff=internal --with-jpeg=internal --with-gif=internal --with-png=internal --with-libz=internal
# make -j 16
# make install

References:

by

Topology optimization using TOSCA for ABAQUS in HPC

Topology optimization using TOSCA for Abaqus is executed by running the Abaqus execution procedure and by providing the name of the parameter file, which, in turn, refers to an Abaqus input file. You can define your optimization model using Abaqus/CAE, and you can use the Optimization Process Manager to create the parameter file and the Abaqus/CAE input file.

https://abaqus-docs.mit.edu/2017/English/SIMACAEEXCRefMap/simaexc-c-topooptimizeproc.htm

by

Testing for license checkout for MATLAB_Distrib_Comp_Engine

If you need to check whether you can check out the distributing computer

[user1@hpc-gekko1 ~]# /usr/local/MATLAB/R2019a/bin/matlab -dmlworker =r "license, exit"
X11 connection rejected because of wrong authentication.
MATLAB is selecting SOFTWARE OPENGL rendering.
======BEGIN LICENSE MANAGER ERROR======
License checkout failed.
License Manager Error -5
Cannot find a license for MATLAB_Distrib_Comp_Engine.
Troubleshoot this issue by visiting:
https://www.mathworks.com/support/lme/R2019a/5

Diagnostic Information:
Feature: MATLAB_Distrib_Comp_Engine
License path: /root/.matlab/R2019a_licenses:/usr/local/MATLAB/R2019a/licenses/license.dat:/usr/local/MATLAB/R2019a
/licenses/network.lic
Licensing error: -5,147. System Error: 2======END LICENSE MANAGER ERROR======

2. Check the “normal MATLAB License Server”

$ /usr/local/MATLAB/R2019a/etc/glnxa64/lmutil lmstat -a -c ../../licenses/network.lic -f MATLAB_Distr_Comp_Engine
lmutil - Copyright (c) 1989-2017 Flexera Software LLC. All Rights Reserved.
Flexible License Manager status on Thu 10/24/2019 12:31

License server status: 27000@xxx.xxx.xxx.xxx
License file(s) on xxx.xxx.xxx.xxx: C:\Program Files\MATLAB\R2019b\etc\license.dat:

xxx.xxx.xxx.xxx: license server UP v11.16.2

Vendor daemon status (on N4SAP0048):

MLM: UP v11.16.2
Feature usage info:
by

Compiling STAR 2.7 with Intel Compiler 2018 on CentOS 7

What is STAR 2.7?
Spliced Transcripts Alignment to a Reference
https://www.ncbi.nlm.nih.gov/pubmed/23104886


Hardware/Software Requirements
x86-64 compatible processors


Manual
https://github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf


Compiling for Source
# Source Intel Compilers

source /usr/local/intel/2018u3/bin/compilervars.sh intel64
source /usr/local/intel/2018u3/impi/2018.3.222/bin64/mpivars.sh intel64
source /usr/local/intel/2018u3/mkl/bin/mklvars.sh intel64
source /usr/local/intel/2018u3/parallel_studio_xe_2018/bin/psxevars.sh intel64
MKLROOT=/usr/local/intel/2018u3/mkl
# wget https://github.com/alexdobin/STAR/archive/2.7.1a.tar.gz
# tar -xvzf 2.7.1a.tar.gz
# cd STAR-2.7.1a/source
# mkdir /usr/local/STAR-2.7.1a
# cp STAR /usr/local/STAR-2.7.1a/
# cp STARlong /usr/local/STAR-2.7.1a/
# cp -r htslib /usr/local/STAR-2.7.1a/
by

Compile LATTE package for LAMMPS using Intel Compilers

Step 1: Download LATTE Source Code from Project

Download or clone the LATTE source code from https://github.com/lanl/LATTE. If you download a zipfile or tarball, unpack the tarball either in this /lib/latte directory or somewhere else on your system.

Step 2: Modify makefile.CHOICES

Modify the makefile.CHOICES according to your system architecture and compilers. Check that the MAKELIB flag is ON in makefile.CHOICES and finally, build the code via the make command

#
# Compilation and link flags for LATTE
#

# Precision - double or single
PRECISION = DOUBLE
#PRECISION = SINGLE

# Make the latte library
# AR and RUNLIB executable default path to compile
# latte as a library (change accordingly)
MAKELIB = ON
AR = /usr/bin/ar cq
RANLIB = /usr/bin/ranlib

# Use PROGRESS and BML libraries
PROGRESS = OFF
PROGRESS_PATH= $(HOME)/qmd-progress/install/lib
BML_PATH= $(HOME)/bml/install/lib

# Use METIS library for graph partitioning
METIS = OFF
METIS_PATH= $(HOME)/metis/metis-5.1.0/install

# GPU available - OFF or ON
GPUOPT = OFF

# Using DBCSR library from cp2k? OFF or ON
DBCSR_OPT = OFF

# Parallelizing over k-points?
MPIOPT = OFF

#
# CPU Fortran options
#

#For GNU compiler:
#FC = mpif90
FC = gfortran
FCL = $(FC)
FFLAGS = -O3 -fopenmp -cpp
#FFLAGS =  -fast -Mpreprocess -mp
LINKFLAG = -fopenmp

#For intel compiler:
FC = ifort
FCL = $(FC)
FFLAGS =  -O3 -fpp -qopenmp
LINKFLAG = -qopenmp
LIB = -mkl=parallel

#GNU BLAS/LAPACK libraries:
#LIB = -L/usr/local/lapack-3.8.0 -llapack -L/usr/local/blas-3.8.0/lib -lblas

#Intel MKL BLAS/LAPACK libraries:
LIB = -Wl,--no-as-needed -L${MKLROOT}/lib/intel64 \
 -lmkl_lapack95_lp64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core \
 -lmkl_gnu_thread -lmkl_core -ldl -lpthread -lm

#Alternative flags for MKL:
#LIB += -mkl=parallel

#Other BLAS/LAPACK vendors:
#LIB = -framework Accelerate
#LIB = -L/usr/projects/hpcsoft/toss2/common/acml/5.3.1/gfortran64/lib -lacml

# Uncomment for coverage
#CVR = OFF
ifeq ($(CVR), ON)
        FFLAGS += -fprofile-arcs -ftest-coverage
        LINKFLAG += -fprofile-arcs -ftest-coverage
endif

ifeq ($(PROGRESS), ON)
        LIB += -L$(PROGRESS_PATH) -lprogress -L$(BML_PATH) -lbml_fortran -lbml
        FFLAGS += -I$(BML_PATH)/../include -I$(PROGRESS_PATH)/../include
endif

ifeq ($(GRAPH), ON)
        LIB += -L$(METIS_PATH)/lib -lmetis
        FFLAGS += -I$(METIS_PATH)/include
endif

#DBCSR_LIB = -L/home/cawkwell/cp2k/lib/cawkwell/popt -lcp2k_dbcsr_lib
#DBCSR_MOD = -I/home/cawkwell/cp2k/obj/cawkwell/popt

#
# GPU options
#

GPU_CUDA_LIB = -L/usr/local/cuda-9.1/lib64 -lcublas -lcudart
GPU_ARCH = sm_20

# make

You should see liblatte.a file.