Compiling Open Tool for Parameter Optimization (otpo)


Taken from Open Tool for Parameter Optimization (otpo) Documentation

OTPO (Open Tool for Parameter Optimizations) is an Open MPI specific tool that is meant to explore the MCA parameter space. In Open MPI, the user can specify at runtime many values for MCA parameters, try e.g. (ompi_info –param all all). Alternatively, to focus on a single aspect of parameters, e.g. the parameters of the OpenIB BTL (ompi_info –param btl openib).

OTPO is a tool that takes in a list of any MCA parameters, with a user specified range of values for those parameters, and for every combination of the MCA parameter values, OTPO executes an MPI job, measuring execution time (the only measurement available right now), bandwidth, etc.. The tests used for the measurements are modular. Right now, OTPO supports

Netpipe
Skampi
NAS Parallel Benchmarks

OTPO outputs a list of the best parameter combinations for a certain test.

The main purpose of OTPO is to explore the effect of the MCA parameters on different machines with different architectures and configurations, and explore the dependencies between the MCA parameters themselves. OTPO is meant to run on the head node of a cluster, and it forks MPI jobs after exporting the current combination of MCA parameters on the nodes.

OTPO is built on top of ADCL (Abstract Data and Communication Library). ADCL is an application level communication library aiming at providing the highest possible performance for application level communication operations in a given execution environment. OTPO uses ADCL to provide the runtime selection logic and choosing the best combination of parameters.


A. Compiling OTPO

Step 1: Make sure your PATH and LD_LIBRARY_PATH has the necessary pathing for the OpenMPI and Compilers

In your .bashrc, it should look something like

export PATH=$PATH:/usr/local/intel/composerxe/bin:/usr/local/openmpi-1.6.5-intel-v12.1.5/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/mpc-0.8.1/lib:/usr/local/intel/composerxe/lib/intel64:/usr/local/intel/composerxe/mkl/lib/intel64:/usr/local/openmpi-1.6.5-intel-v12.1.5/lib

Step 2: Download and compile ADCL within OTPO

# wget http://www.open-mpi.org/software/otpo/v1.0/downloads/otpo-1.0.tar.gz
# tar -zxvf otpo-1.0.tar.gz
# cd otpo
# cd ADCL
# ./configure --prefix=/usr/local/ADCL
# make
# make install
# cp /root/otpo/ADCL/include/*.h /usr/local/ADCL

After the compilation, you should see libadcl.a inside the lib folder of ADCL. Remember to copy the ADCL headers to the /usr/local/ADCL/include

Step 3: Compile OTPO

# ./autogen.sh
# ./configure --prefix=/usr/local/otpo-1.0 --with-adcl-dir=/usr/local/ADCL
# make
# make install

One thought on “Compiling Open Tool for Parameter Optimization (otpo)

  1. When i compile OTPO the following shows:
    /usr/bin/ld: adcl_setup.o: undefined reference to symbol ‘floor@@GLIBC_2.2.5′
    /usr/lib64/libm.so.6: error adding symbols: DSO missing from command line
    collect2: error: ld returned 1 exit status
    what should i do.
    The full screen output
    [asc02@gpu3 otpo]$ ./configure –prefix=/public/home/asc02/yangxf/local/otpo –with-adcl-dir=/public/home/asc02/yangxf/local/ADCL
    Configuring OTPO
    checking for a BSD-compatible install… /usr/bin/install -c
    checking whether build environment is sane… yes
    checking for a thread-safe mkdir -p… /usr/bin/mkdir -p
    checking for gawk… gawk
    checking whether make sets $(MAKE)… yes
    checking whether make supports nested variables… yes
    checking whether make supports nested variables… (cached) yes
    checking for otpo version… 1.0.1a1-1
    checking build system type… x86_64-unknown-linux-gnu
    checking host system type… x86_64-unknown-linux-gnu
    checking for mpicc… yes
    checking for gcc… mpicc
    checking whether the C compiler works… yes
    checking for C compiler default output file name… a.out
    checking for suffix of executables…
    checking whether we are cross compiling… no
    checking for suffix of object files… o
    checking whether we are using the GNU C compiler… yes
    checking whether mpicc accepts -g… yes
    checking for mpicc option to accept ISO C89… none needed
    checking for style of include used by make… GNU
    checking dependency style of mpicc… gcc3
    checking for ranlib… ranlib
    checking how to run the C preprocessor… mpicc -E
    checking for grep that handles long lines and -e… /usr/bin/grep
    checking for egrep… /usr/bin/grep -E
    checking for ANSI C header files… yes
    checking for sys/types.h… yes
    checking for sys/stat.h… yes
    checking for stdlib.h… yes
    checking for string.h… yes
    checking for memory.h… yes
    checking for strings.h… yes
    checking for inttypes.h… yes
    checking for stdint.h… yes
    checking for unistd.h… yes
    checking mpi.h usability… yes
    checking mpi.h presence… yes
    checking for mpi.h… yes
    checking for MPI_Initialized… yes
    checking for ADCL main directory… /public/home/asc02/yangxf/local/ADCL
    checking for ADCL lib directory… /public/home/asc02/yangxf/local/ADCL/lib
    checking for ADCL library name… libadcl.a
    checking if we want MPI_LEAVE_PINNED… yes
    checking if we want MPI_PAFFINITY_ALONE… yes
    checking ADCL.h usability… yes
    checking ADCL.h presence… yes
    checking for ADCL.h… yes
    checking for ADCL_Init… no
    checking that generated files are newer than configure… done
    configure: creating ./config.status
    config.status: creating Makefile
    config.status: creating otpo_config.h
    config.status: executing depfiles commands
    [asc02@gpu3 otpo]$ make
    cd . && /bin/sh ./config.status otpo_config.h
    config.status: creating otpo_config.h
    config.status: otpo_config.h is unchanged
    make all-recursive
    make[1]: Entering directory `/public/home/asc02/yangxf/softsource/otpo’
    make[2]: Entering directory `/public/home/asc02/yangxf/softsource/otpo’
    CC generate_input_file.o
    CC parse.o
    CC otpo.o
    CC adcl_setup.o
    CC output.o
    CCLD otpo
    /usr/bin/ld: adcl_setup.o: undefined reference to symbol ‘floor@@GLIBC_2.2.5′
    /usr/lib64/libm.so.6: error adding symbols: DSO missing from command line
    collect2: error: ld returned 1 exit status
    make[2]: *** [otpo] Error 1
    make[2]: Leaving directory `/public/home/asc02/yangxf/softsource/otpo’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/public/home/asc02/yangxf/softsource/otpo’
    make: *** [all] Error 2
    [asc02@gpu3 otpo]$ make clean
    make[1]: Entering directory `/public/home/asc02/yangxf/softsource/otpo’
    test -z “otpo” || rm -f otpo
    test -z “*~” || rm -f *~
    rm -f *.o
    make[1]: Leaving directory `/public/home/asc02/yangxf/softsource/otpo’
    [asc02@gpu3 otpo]$ make
    cd . && /bin/sh ./config.status otpo_config.h
    config.status: creating otpo_config.h
    config.status: otpo_config.h is unchanged
    make all-recursive
    make[1]: Entering directory `/public/home/asc02/yangxf/softsource/otpo’
    make[2]: Entering directory `/public/home/asc02/yangxf/softsource/otpo’
    cd . && /bin/sh ./config.status otpo_config.h
    config.status: creating otpo_config.h
    config.status: otpo_config.h is unchanged
    CC generate_input_file.o
    CC parse.o
    CC otpo.o
    CC adcl_setup.o
    CC output.o
    CCLD otpo
    /usr/bin/ld: adcl_setup.o: undefined reference to symbol ‘floor@@GLIBC_2.2.5′
    /usr/lib64/libm.so.6: error adding symbols: DSO missing from command line
    collect2: error: ld returned 1 exit status
    make[2]: *** [otpo] Error 1
    make[2]: Leaving directory `/public/home/asc02/yangxf/softsource/otpo’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/public/home/asc02/yangxf/softsource/otpo’
    make: *** [all] Error 2

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.