Testing compatibility of Gromacs with CUDA Drivers

To test whether you have compiled your GROMACS correctly with the CUDA drivers and runtime. You can use the command

% gmx_mpi --version

You should see

GPU support: CUDA
.....
.....
CUDA driver: 10.10
CUDA runtime: 10.10

Regression Test Errors during Gromacs Compilation

If you are compiling Gromacs like Compiling Gromacs-2019.3 with Intel MKL and CUDA and encounter errors similar to the ones below:

Downloading: http://gerrit.gromacs.org/download/regressiontests-2020.3.tar.gz -- [download 100% complete] CMake Error at tests/CMakeLists.txt:57 (message): error: downloading 'http://gerrit.gromacs.org/download/regressiontests-2020.3.tar.gz' failed status_code: 7 status_string: 

"Couldn't connect to server" log: Trying 130.237.25.133... TCP_NODELAY set Connected to gerrit.gromacs.org (130.237.25.133) port 80 (#0) GET /download/regressiontests-2020.3.tar.gz HTTP/1.1 Host: gerrit.gromacs.org User-Agent: curl/7.54.1 Accept: */* HTTP/1.1 302 Found Date: Mon, 28 Dec 2020 02:26:13 GMT Server: Apache/2.4.18 (Ubuntu) Location: ftp://ftp.gromacs.org/regressiontests/regressiontests-2020.3.tar.gz Content-Length: 335 Content-

Type: text/html; charset=iso-8859-1 Ignoring the response-body [335 bytes data] Connection #0 to host gerrit.gromacs.org left intact Issue another request to this URL: 'ftp://ftp.gromacs.org/regressiontests/regressiontests-2020.3.tar.gz' Trying 130.237.11.165... TCP_NODELAY set 

Connected to ftp.gromacs.org (130.237.11.165) port 21 (#1) 220 Welcome to the GROMACS FTP service. USER anonymous 331 Please specify the password. PASS ftp@example.com 230 Login successful. PWD 257 "/" is the current directory Entry path is '/' CWD regressiontests ftp_perform ends with SECONDARY: 0 250 Directory successfully changed. ..... ..... Connecting to 130.237.11.165 (130.237.11.165) port 10087 connect to 130.237.11.165 port 21 failed: Connection timed out Failed to connect to ftp.gromacs.org port 21: Connection timed out Closing connection 1

To workaround,

Step 1: Download the Relevant Test from https://ftp.gromacs.org/regressiontests/

Step 2: Untar the regression test to the Gromacs Source Directory

% tar -zxvf  egressiontests-2019.6.tar.gz

Step 3: Create a script file

% touch gromacs_gpgpu.sh

Include -DREGRESSIONTEST_DOWNLOAD=OFF -DREGRESSIONTEST_PATH=../regressiontests-2019.6

To see the whole makefile, see

CC=mpicc CXX=mpicxx cmake .. -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DGMX_MPI=on -DGMX_FFT_LIBRARY=mkl \
-DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-2019.3_intel18_mkl_cuda11.1 -DREGRESSIONTEST_DOWNLOAD=OFF -DREGRESSIONTEST_PATH=../regressiontests-2019.6 \
-DCMAKE_C_FLAGS:STRING="-cc=icc -O3 -xHost -ip" \
-DCMAKE_CXX_FLAGS:STRING="-cxx=icpc -O3 -xHost -ip -I/usr/local/intel/2018u3/compilers_and_libraries_2018.3.222/linux/mpi/intel64/include/" \
-DGMX_GPU=on \
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.1 \
-DCMAKE_BUILD_TYPE=Release \
-DCUDA_HOST_COMPILER:FILEPATH=/usr/local/intel/2018u3/compilers_and_libraries_2018.3.222/linux/bin/intel64/icpc

 

Compiling Gromacs-2019.3 with Intel 2018 MKL and AVX-512

Prerequisites

GCC-6.5 Compilers and associates libraries
m4-1.4.18
mpfr-3.1.4
cmake-3.15.1
gmp-6.1.0
mpc-1.0.3

Intel Compilers and Prerequisites

source /usr/local/intel/2018u3/bin/compilervars.sh intel64
source /usr/local/intel/2018u3/impi/2018.3.222/bin64/mpivars.sh intel64
source /usr/local/intel/2018u3/mkl/bin/mklvars.sh intel64
source /usr/local/intel/2018u3/parallel_studio_xe_2018/bin/psxevars.sh intel64
MKLROOT=/usr/local/intel/2018u3/mkl

Create a setup file

touch gromacs_cpu.sh

Put the following into the gromacs_cpu.sh

CC=mpicc CXX=mpicxx cmake .. -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx
-DGMX_MPI=on -DGMX_FFT_LIBRARY=mkl -DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-2019.3_intel18_mkl 
-DREGRESSIONTEST_DOWNLOAD=ON -DCMAKE_C_FLAGS:STRING="-cc=icc -O3 -xHost -ip" 
-DCMAKE_CXX_FLAGS:STRING="-cxx=icpc -O3 -xHost -ip" -DGMX_GPU=off -DCMAKE_BUILD_TYPE=Release -DGMX_SIMD=AVX_512

SIMD Support can be found at http://manual.gromacs.org/current/install-guide/index.html#simd-support

./gromacs_cpu.sh
make
make install

Testing and Verification

$ source /your/installation/prefix/here/bin/GMXRC
./gmxtest.pl all -np 2

Compiling Gromacs-2019.3 with Intel MKL and CUDA

Prerequisites

GCC-6.5 Compilers and associates libraries
m4-1.4.18
mpfr-3.1.4
cmake-3.15.1
gmp-6.1.0
mpc-1.0.3

Intel Compilers and Prerequisites

source /usr/local/intel/2018u3/bin/compilervars.sh intel64
source /usr/local/intel/2018u3/impi/2018.3.222/bin64/mpivars.sh intel64
source /usr/local/intel/2018u3/mkl/bin/mklvars.sh intel64
source /usr/local/intel/2018u3/parallel_studio_xe_2018/bin/psxevars.sh intel64
MKLROOT=/usr/local/intel/2018u3/mkl

Create a setup file

touch gromacs_gpgpu.sh

Put the following into the gromacs_cpu.sh

CC=mpicc CXX=mpicxx cmake .. -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DGMX_MPI=on -DGMX_FFT_LIBRARY=mkl
-DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-2019.3_intel18_mkl_cuda10.1 -DREGRESSIONTEST_DOWNLOAD=ON
-DCMAKE_C_FLAGS:STRING="-cc=icc -O3 -xHost -ip"
-DCMAKE_CXX_FLAGS:STRING="-cxx=icpc -O3 -xHost -ip -I/usr/local/intel/2018u3/compilers_and_libraries_2018.3.222/linux/mpi/intel64/include/" 
-DGMX_GPU=on 
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.1
-DCMAKE_BUILD_TYPE=Release
-DCUDA_HOST_COMPILER:FILEPATH=/usr/local/intel/2018u3/compilers_and_libraries_2018.3.222/linux/bin/intel64/icpc
./gromacs_gpgpu.sh
make
make install

Testing and Verification

$ source /your/installation/prefix/here/bin/GMXRC
./gmxtest.pl all -np 2

Compiling Gromacs-2019.3 with Intel 2018 and MKL

Prerequisites

GCC-6.5 Compilers and associates libraries
m4-1.4.18
mpfr-3.1.4
cmake-3.15.1
gmp-6.1.0
mpc-1.0.3

Intel Compilers and Prerequisites

source /usr/local/intel/2018u3/bin/compilervars.sh intel64
source /usr/local/intel/2018u3/impi/2018.3.222/bin64/mpivars.sh intel64
source /usr/local/intel/2018u3/mkl/bin/mklvars.sh intel64
source /usr/local/intel/2018u3/parallel_studio_xe_2018/bin/psxevars.sh intel64
MKLROOT=/usr/local/intel/2018u3/mkl

Create a setup file

touch gromacs_cpu.sh

Put the following into the gromacs_cpu.sh

CC=mpicc CXX=mpicxx cmake .. -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx
-DGMX_MPI=on -DGMX_FFT_LIBRARY=mkl -DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-2019.3_intel18_mkl 
-DREGRESSIONTEST_DOWNLOAD=ON -DCMAKE_C_FLAGS:STRING="-cc=icc -O3 -xHost -ip" 
-DCMAKE_CXX_FLAGS:STRING="-cxx=icpc -O3 -xHost -ip" -DGMX_GPU=off -DCMAKE_BUILD_TYPE=Release
./gromacs_cpu.sh
make
make install

Testing and Verification

$ source /your/installation/prefix/here/bin/GMXRC
./gmxtest.pl all -np 2

Compiling Gromacs 5.0.4 on CentOS 6

Compiling Gromacs has never been easier using the cmake. There are a few assumptions.

  1. Use MKL and Intel Compilers
  2. Use OpenMPI  as the MPI-of-choice. The necessary PATH and LD_LIBRARY_PATH have been placed in .bashrc
  3. We will use SINGLE precision for speed used MDRUN and MPI Flags

Here is my configuration file using Intel Compilers

# tar xfz gromacs-5.0.4.tar.gz
# cd gromacs-5.0.4
# mkdir build
# cd build
# /usr/local/cmake-3.1.3/bin/cmake 
-DGMX_BUILD_OWN_FFTW=ON \
-DREGRESSIONTEST_DOWNLOAD=OFF \ 
-DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-5.0.4 \
-DGMX_MPI=on \
-DGMX_FFT_LIBRARY=mkl \
-DMKL_LIBRARIES="/usr/local/intel/mkl/lib/intel64" \
-DMKL_INCLUDE_DIR="/usr/local/intel/mkl/include" \
-DGMX_DOUBLE=on \
-DGMX_BUILD_MDRUN_ONLY=off \ 
-DCMAKE_C_COMPILER=mpicc \ 
-DCMAKE_CXX_COMPILER=mpicxx
# make
# make check
# sudo make install
# source /usr/local/gromacs/bin/GMXRC

Installation Flags

–   -DCMAKE_C_COMPILER=xxx equal to the name of the C99 compiler you
    wish to use (or the environment variable CC)
–   -DCMAKE_CXX_COMPILER=xxx equal to the name of the C++98 compiler you
    wish to use (or the environment variable CXX)
–   -DGMX_MPI=on to build using an MPI wrapper compiler
–   -DGMX_GPU=on to build using nvcc to run with an NVIDIA GPU
–   -DGMX_SIMD=xxx to specify the level of SIMD support of the node on
    which mdrun will run
–   -DGMX_BUILD_MDRUN_ONLY=on to build only the mdrun binary, e.g. for
    compute cluster back-end nodes
–   -DGMX_DOUBLE=on to run GROMACS in double precision (slower, and not
    normally useful)
–   -DCMAKE_PREFIX_PATH=xxx to add a non-standard location for CMake to
    search for libraries
–   -DCMAKE_INSTALL_PREFIX=xxx to install GROMACS to a non-standard
    location (default /usr/local/gromacs)
–   -DBUILD_SHARED_LIBS=off to turn off the building of shared libraries
–   -DGMX_FFT_LIBRARY=xxx to select whether to use fftw, mkl or fftpack
    libraries for FFT support
–   -DCMAKE_BUILD_TYPE=Debug to build GROMACS in debug mode

Compiling and Installing Gromacs 4.6.2 with OpenMPI and Intel on CentOS 6

Step 1: Compiling cmake
To compile Gromacs 4.6.2, first you need to compile cmake 2.8 and above. At this point in writing, the current version is cmake 2.8.11.2

# tar -zxvf cmake-2.8.11.2.tar.gz
# cd cmake-2.8.11.2
# ./configure --prefix=/usr/local/cmake-2.8.11.2
# make
# make install

Step 2: Installing OpenMPI
See Blog Entry Building OpenMPI with Intel Compilers

Step 3: Installing FFTW (Single and Double Precision)
See Blog Entry Compiling and installing FFTW 3.3.3.

Step 4: Compiling Gromacs 4.6.2 (Single-Precision with Intel and OpenMPI)

# tar -zxvf gromacs-4.6.2.tar.gz
# cd gromacs-4.6.2
# mkdir build-cmake
# cd build-cmake
# CC=icc CMAKE_PREFIX_PATH=/usr/local/fftw-3.3.3-single/ /usr/local/cmake-2.8.11.2/bin/cmake .. 
-DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-4.6.2-single-intel -DGMX_X11=OFF -DGMX_MPI=ON 
-DGMX_PREFER_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF
# make
# make install

Step 5: Compiling Gromacs 4.6.2 (Double Precision with Intel and OpenMPI)

# tar -zxvf gromacs-4.6.2.tar.gz
# cd gromacs-4.6.2
# mkdir build-cmake
# cd build-cmake
# CC=icc CMAKE_PREFIX_PATH=/usr/local/fftw-3.3.3-single/ /usr/local/cmake-2.8.11.2/bin/cmake .. 
-DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-4.6.2-single-intel -DGMX_X11=OFF -DGMX_MPI=ON 
-DGMX_PREFER_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF -DGMX_DOUBLE=ON
# make 
# make install

For more information,

  1. Compiling Gromacs 4.6 and PLUMED from Source
  2. Gromacs Installation instructions