Day: October 12, 2018

by

Compiling with NWChem-6.8 with Intel MPI 2018u3

Here is a write-up of my computing platform and applications:

  1. NWChem 6.8 (14 December 2017)
  2. Intel Compilers, IMPI and MKL (2018 U3)
  3. Infiniband Interconnect (OFED-4.3-1.0.1)
  4. CentOS 7.4 (x86_64)

Step 1: First thing first, source the intel components setting from

# source /usr/local/intel/2018u3/bin/compilervars.sh intel64
# source /usr/local/intel/2018u3/impi/2018.3.222/bin64/mpivars.sh intel64
# source /usr/local/intel/2018u3/mkl/bin/mklvars.sh intel64
# source /usr/local/intel/2018u3/parallel_studio_xe_2018/bin/psxevars.sh intel64
export NWCHEM_TOP=/usr/local/software/nwchem-6.8/nwchem-6.8
export NWCHEM_MODULES=pnnl
export NWCHEM_TARGET=LINUX64
export NWCHEM_LONG_PATHS=y
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export PYTHONLIBTYPE=so
export USE_PYTHON64=y
export USE_NOFSCHECK=y
export TCGRSH=/usr/bin/ssh
export LARGE_FILES=y
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC="/usr/local/intel/2018u3/impi/2018.3.222/intel64"
export MPI_INCLUDE="/usr/local/intel/2018u3/impi/2018.3.222/intel64/include/gfortran/5.1.0 -I/usr/local/intel/2018u3/impi/2018.3.222/intel64/include"
export MPI_LIB="/usr/local/intel/2018u3/impi/2018.3.222/intel64/lib/release_mt -L/usr/local/intel/2018u3/impi/2018.3.222/intel64/lib"
export LIBMPI="-lmpifort -lmpi -lmpigi -ldl -lrt -lpthread"
export USE_OPENMP=y
export MKLROOT=/usr/local/intel/2018u3/mkl
export MKLLIB="${MKLROOT}/lib/intel64"
export MKLINC="${MKLROOT}/include"
export HAS_BLAS=y
export BLAS_SIZE=8
export BLASOPT="-L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl"
export LAPACK_SIZE=8
export LAPACK_LIB="$BLASOPT"
export LAPACK_LIBS="$BLASOPT"
export LAPACKOPT="$BLASOPT"
export USE_SCALAPACK=y
export SCALAPACK_SIZE=8
export SCALAPACK="-L${MKLROOT}/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_ilp64 -liomp5 -lpthread -lm -ldl"
export SCALAPACK_LIB="$SCALAPACK"
export SCALAPACK_LIBS="$SCALAPACK"
export CC=icc
export FC=ifort
export USE_64TO32=y

cd $NWCHEM_TOP/src
#make realclean
make nwchem_config
make 64_to_32
make CC=icc FC=ifort FOPTIMIZE=-O3
cd $NWCHEM_TOP/src/tools
make CC=icc FC=ifort FOPTIMIZE=-O3 version
make CC=icc FC=ifort FOPTIMIZE=-O3
cd $NWCHEM_TOP/src
make CC=icc FC=ifort FOPTIMIZE=-O3 link

General Site Installation

Determine the local storage path for the install files. (e.g., /usr/local/NWChem).
Make directories

Determine the local storage path for the install files. (e.g., /usr/local/NWChem).
Make directories

# mkdir /usr/local/nwchem-6.8
# mkdir /usr/local/nwchem-6.8/bin
# mkdir /usr/local/nwchem-6.8/data

Copy binary

# cp $NWCHEM_TOP/bin/ /usr/local/nwchem-6.8/bin
# cd /usr/local/nwchem-6.8/bin
# chmod 755 nwchem

Copy libraries

# cd $NWCHEM_TOP/src/basis
# cp -r libraries /usr/local/nwchem-6.8/data

# cd $NWCHEM_TOP/src/
# cp -r data /usr/local/nwchem-6.8

# cd $NWCHEM_TOP/src/nwpw
# cp -r libraryps /usr/local/nwchem-6.8/data

The Final Lap (From Compiling NWChem)

Each user will need a .nwchemrc file to point to these default data files. A global one could be put in /usr/local/nwchem-6.8/data and a symbolic link made in each users $HOME directory is probably the best plan for new installs. Users would have to issue the following command prior to using NWChem: ln -s /usr/local/nwchem-6.8/data/default.nwchemrc $HOME/.nwchemrc

Contents of the default.nwchemrc file based on the above information should be:

nwchem_basis_library /usr/local/nwchem-6.8/data/libraries/
nwchem_nwpw_library /usr/local/nwchem-6.8/data/libraryps/
ffield amber
amber_1 /usr/local/nwchem-6.8/data/amber_s/
amber_2 /usr/local/nwchem-6.8/data/amber_q/
amber_3 /usr/local/nwchem-6.8/data/amber_x/
amber_4 /usr/local/nwchem-6.8/data/amber_u/
spce    /usr/local/nwchem-6.8/data/solvents/spce.rst
charmm_s /usr/local/nwchem-6.8/data/charmm_s/
charmm_x /usr/local/nwchem-6.8/data/charmm_x/

References:

  1. Compiling NWCHEM