Compiling and Installing Meep-1.2.1 on CentOS 6 and OpenMPI

Meep (or MEEP) is a free finite-difference time-domain (FDTD) simulation software package developed at MIT to model electromagnetic systems, along with our MPB eigenmode package. The latest official version is 1.2 and can be found at  Download Page for Meep

But there is a Lapack linking problem for 1.2 which is explained in Error when compiling Meep-1.2 on CentOS. It is strongly recommended to use the pre-release Meep 1.2.1 found at http://jdj.mit.edu/~stevenj/meep-1.2.1.tar.gz

Before you compile Meep 1.2.1, you need to first compile the libctl library. Compiling the libctl library is quite straightforward. After downloading,

Step 1: Compiling libctl-3.2.1

# tar -zxvf libctl-3.2.1.tar.gz
# cd libctl-3.2.1
# ./configure --prefix=/usr/local/libctl-3.2.1
# make -j8
# make install

Step 2: Other Prerequisites include guile and guile-devel. Do make sure you install these 2 packages which can be done

# yum install guile guile-devel

Step 3: Compiling OpenMPI,
Do look at Compiling OpenMPI 1.6.5 with Intel 12.1.5 on CentOS 6

Step 4: Compiling meep-1.2.1

# tar -zxvf meep-1.2.1.tar.gz
# cd meep-1.2.1
#  ./configure --prefix=/usr/local/meep-1.2.1 \
--with-libctl=/usr/local/libctl-3.2.1/share/libctl/ \
LDFLAGS=-L/usr/local/libctl-3.2.1/lib \
CPPFLAGS=-I/usr/local/libctl-3.2.1/include \
--with-mpi

References:

  1. Undefined reference to `dgetrf_’ error when compiling Meep-1.2 on CentOS
  2. crtbegin.o: No such file: No such file or directory Error when compiling Meep-1.2 on CentOS
  3. My opinion: Compiling Meep

Compiling GNU Scientific Library (GSL) gsl-1.16 on CentOS 6

The GNU Scientific Library (GSL) is a numerical library for C and C++ programmers. It is free software under the GNU General Public License.

The library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting. There are over 1000 functions in total with an extensive test suite.

Step 1: The current version of GSL is gsl-1.16.tar.gz

Step 2: You may want to use the latest GCC 4.8.1 to compile. For more information on how to compile GCC 4.8.1, see Compiling GNU 4.8.1 on CentOS 6. This compilation will help to fix all the components required for gsl-1.16

Step 3: After packing gsl, To compile

# cd /root/gsl-1.15/
# mkdir build-gsl
# cd build-gsl
# ../configure --prefix=/usr/local/gsl-1.16/
# make 
# make install

Compiling and installing FFTW 3.3.3 with OpenMPI and OpenMP

This Blog entry is an extension of the Compiling and installing FFTW 3.3.3

To Compile FFTW 3.3.3 single precision with OpenMPI, make sure you compile and set your path for Intel and OpenMPI. You may want to get more information from Compiling OpenMPI 1.6.5 with Intel 12.1.5 on CentOS 6

Step 1: Compiling FFTW 3.3.3 (Single Precision) with OpenMPI and OpenMP

After unpacking FFTW 3.3.3, you may want to use the flags

# ./configure CC=icc 
--enable-float --enable-threads --enable-openmp \
--enable-mpi MPICC=mpicc \
LDFLAGS=-L/usr/local/openmpi/intel/lib CPPFLAGS=-I/usr/local/openmpi/intel/include \ 
--prefix=/usr/local/fftw-3.3.3-single
# make -j8
# make install

Inside /usr/local/fftw-3.3.3-single/lib, you should see at least the files below

libfftw3f.a
libfftw3f_mpi.a
libfftw3f_omp.a
libfftw3f_threads.a
....
....

Step 2: Compiling FFTW 3.3.3 (Double Precision) with OpenMPI and OpenMP

# ./configure CC=icc 
--enable-threads --enable-openmp \
--enable-mpi MPICC=mpicc \
LDFLAGS=-L/usr/local/openmpi/intel/lib CPPFLAGS=-I/usr/local/openmpi/intel/include \ 
--prefix=/usr/local/fftw-3.3.3-single
# make -j8
# make install

Inside /usr/local/fftw-3.3.3-double/lib, you should see at least the files below

libfftw3.a
libfftw3_mpi.a
libfftw3_omp.a
libfftw3_threads.a
....
....

Compiling Open Tool for Parameter Optimization (otpo)

Taken from Open Tool for Parameter Optimization (otpo) Documentation

OTPO (Open Tool for Parameter Optimizations) is an Open MPI specific tool that is meant to explore the MCA parameter space. In Open MPI, the user can specify at runtime many values for MCA parameters, try e.g. (ompi_info –param all all). Alternatively, to focus on a single aspect of parameters, e.g. the parameters of the OpenIB BTL (ompi_info –param btl openib).

OTPO is a tool that takes in a list of any MCA parameters, with a user specified range of values for those parameters, and for every combination of the MCA parameter values, OTPO executes an MPI job, measuring execution time (the only measurement available right now), bandwidth, etc.. The tests used for the measurements are modular. Right now, OTPO supports

Netpipe
Skampi
NAS Parallel Benchmarks

OTPO outputs a list of the best parameter combinations for a certain test.

The main purpose of OTPO is to explore the effect of the MCA parameters on different machines with different architectures and configurations, and explore the dependencies between the MCA parameters themselves. OTPO is meant to run on the head node of a cluster, and it forks MPI jobs after exporting the current combination of MCA parameters on the nodes.

OTPO is built on top of ADCL (Abstract Data and Communication Library). ADCL is an application level communication library aiming at providing the highest possible performance for application level communication operations in a given execution environment. OTPO uses ADCL to provide the runtime selection logic and choosing the best combination of parameters.


A. Compiling OTPO

Step 1: Make sure your PATH and LD_LIBRARY_PATH has the necessary pathing for the OpenMPI and Compilers

In your .bashrc, it should look something like

export PATH=$PATH:/usr/local/intel/composerxe/bin:/usr/local/openmpi-1.6.5-intel-v12.1.5/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/mpc-0.8.1/lib:/usr/local/intel/composerxe/lib/intel64:/usr/local/intel/composerxe/mkl/lib/intel64:/usr/local/openmpi-1.6.5-intel-v12.1.5/lib

Step 2: Download and compile ADCL within OTPO

# wget http://www.open-mpi.org/software/otpo/v1.0/downloads/otpo-1.0.tar.gz
# tar -zxvf otpo-1.0.tar.gz
# cd otpo
# cd ADCL
# ./configure --prefix=/usr/local/ADCL
# make
# make install
# cp /root/otpo/ADCL/include/*.h /usr/local/ADCL

After the compilation, you should see libadcl.a inside the lib folder of ADCL. Remember to copy the ADCL headers to the /usr/local/ADCL/include

Step 3: Compile OTPO

# ./autogen.sh
# ./configure --prefix=/usr/local/otpo-1.0 --with-adcl-dir=/usr/local/ADCL
# make
# make install