OTPO (Open Tool for Parameter Optimizations) is an Open MPI specific tool that is meant to explore the MCA parameter space. In Open MPI, the user can specify at runtime many values for MCA parameters, try e.g. (ompi_info –param all all). Alternatively, to focus on a single aspect of parameters, e.g. the parameters of the OpenIB BTL (ompi_info –param btl openib).
OTPO is a tool that takes in a list of any MCA parameters, with a user specified range of values for those parameters, and for every combination of the MCA parameter values, OTPO executes an MPI job, measuring execution time (the only measurement available right now), bandwidth, etc.. The tests used for the measurements are modular. Right now, OTPO supports
OTPO outputs a list of the best parameter combinations for a certain test.
The main purpose of OTPO is to explore the effect of the MCA parameters on different machines with different architectures and configurations, and explore the dependencies between the MCA parameters themselves. OTPO is meant to run on the head node of a cluster, and it forks MPI jobs after exporting the current combination of MCA parameters on the nodes.
OTPO is built on top of ADCL (Abstract Data and Communication Library). ADCL is an application level communication library aiming at providing the highest possible performance for application level communication operations in a given execution environment. OTPO uses ADCL to provide the runtime selection logic and choosing the best combination of parameters.
A. Compiling OTPO
Step 1: Make sure your PATH and LD_LIBRARY_PATH has the necessary pathing for the OpenMPI and Compilers
In your .bashrc, it should look something like
export PATH=$PATH:/usr/local/intel/composerxe/bin:/usr/local/openmpi-1.6.5-intel-v12.1.5/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/mpc-0.8.1/lib:/usr/local/intel/composerxe/lib/intel64:/usr/local/intel/composerxe/mkl/lib/intel64:/usr/local/openmpi-1.6.5-intel-v12.1.5/lib
Step 2: Download and compile ADCL within OTPO
# wget http://www.open-mpi.org/software/otpo/v1.0/downloads/otpo-1.0.tar.gz # tar -zxvf otpo-1.0.tar.gz # cd otpo # cd ADCL # ./configure --prefix=/usr/local/ADCL # make # make install # cp /root/otpo/ADCL/include/*.h /usr/local/ADCL
After the compilation, you should see libadcl.a inside the lib folder of ADCL. Remember to copy the ADCL headers to the /usr/local/ADCL/include
Step 3: Compile OTPO
# ./autogen.sh # ./configure --prefix=/usr/local/otpo-1.0 --with-adcl-dir=/usr/local/ADCL # make # make install