GLIBCXX_3.4.25 and cannot support c++17.GLIBCXX Issues

If you have compiled a new appication in an updated GCC-12.3 and still wondering why the Error still surfacing like the message below when you run the binary that was compiled in the GCC-12.3 and not the system GCC

GLIBCXX version in Wildfly using “strings /usr/lib64/libstdc++.so.6 | grep GLIBCXX” and found the latest version in Cluster is only GLIBCXX_3.4.25 and cannot support c++17.

First step in troubleshooting is really to find out what the binary used for its Shared Object Dependencies.

$ ldd main_fcc
./main_fcc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./main_fcc)
./main_fcc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./main_fcc)
linux-vdso.so.1 (0x00007fffffbd1000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f4827ea1000)
libm.so.6 => /lib64/libm.so.6 (0x00007f4827b1f000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f48278e7000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f48276cf000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f48274af000)
libc.so.6 => /lib64/libc.so.6 (0x00007f48270ea000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4828236000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f4826ee6000)

For the above example, “libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f4827ea1000)”, the application seems to be pointing back to the system libraries rather than the libraries you are using. You many want to “module load …….” if you are using Module Environment or configure the PATH, LD_LIBRARY_PATH, MANPATH and CPLUS_INCLUDE_PATH of the GCC you . In the end, the Shared Object Dependencies should be pointing to

$ ldd main_fcc
linux-vdso.so.1 (0x00007fffffbd1000)
libstdc++.so.6 => /usr/local/gnu/gcc-12.3/libstdc++.so.6 (0x00007f4827ea1000)
libm.so.6 => /usr/local/gnu/gcc-12.3/libm.so.6 (0x00007f4827b1f000)
libgomp.so.1 => /usr/local/gnu/gcc-12.3/libgomp.so.1 (0x00007f48278e7000)
libgcc_s.so.1 => /usr/local/gnu/gcc-12.3/libgcc_s.so.1 (0x00007f48276cf000)
libpthread.so.0 => /usr/local/gnu/gcc-12.3/libpthread.so.0 (0x00007f48274af000)
libc.so.6 => /usr/local/gnu/gcc-12.3/libc.so.6 (0x00007f48270ea000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4828236000)
libdl.so.2 => /usr/local/gnu/gcc-12.3/libdl.so.2 (0x00007f4826ee6000)

We have experience in VS Code Environment, where the Shared Object Dependencies was observed to be pointing back to the System Libraries even when using Environment Modules. Then it is best to put in .bashrc something the one below.

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias ls='ls --color=auto'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

module load gnu/gcc-12.3
module load gnu/gdb-14.2

# User specific aliases and functions

Compiling OpenMPI-4.1.5 for ROCEv2 with GNU-8.5

https://docs.open-mpi.org/en/v5.0.x/release-notes/networks.html

Prerequisites 1

First thing first, You may want to check whether you are using RoCE. Do take a look at Installing RoCE using Mellanox (Nvidia) OFED package

Prerequisites 2

Do check whether you have ucx. You can do a dnf install

# dnf install ucx ucx-devel

Alternatively, you can do a manual install. For information on how to install, do take a look at http://openucx.org/wp-content/uploads/UCX_install_guide.pdf

# wget https://github.com/openucx/ucx/releases/download/v1.4.0/ucx-1.4.0.tar.gz
$ tar xzf ucx-1.4.0.tar.gz
$ cd ucx-1.4.0
$ ./contrib/configure-release --prefix=/usr/local/ucx-1.4.0
$ make -j8 
$ make install

Prerequisites 3

Make sure you have install GNU and GNU-C++. This can be done easily using the

# dnf install gcc-c++ gcc

Step 1: Download the OpenMPI package

You can go to OpenMPI to download the latest package at (https://www.open-mpi.org/software/ompi/v4.1/). The latest one at the point of writing is OpenMPI-4.1.

Step 2: Compile the Package

$ ./configure --prefix=/usr/local/openmpi-4.1.5 --enable-mpi-cxx --with-devel-headers --with-ucx --with-verbs --with-slurm=no
$ make && make install

Step 3: To run the MPIRUN using ROCE, do the following.

You may want to see Network Support Information on OpenMPI

$ mpirun --np 12 --hostfile path/to/hostfile --mca pml ucx -x -x UCX_NET_DEVICES=mlx5_0:1 ........

References:

  1. Setting up a RoCE cluster
  2. OpenMPI – Network Support
  3. How do I run Open MPI over RoCE? (UCX PML)

Compiling glibc-2.29 at CentOS-7

Step 1: Download the glibc

To Download the glibc-2.29, do download at https://ftp.gnu.org/gnu/glibc/

Step 2: Compile and Build the glibc libraries

# tar zxvf glibc-2.29.tar.gz
# cd glibc-2.29
# mkdir build
# cd build

Step 3: Compile and install

# ../configure --prefix=/usr/local/glibc-2.29
# make -j8
# make install

Step 4: Errors encountered

.....
checking version of ld... 2.27, ok
checking for gnumake... no
checking for gmake... gmake
checking version of gmake... 3.82, bad
checking for gnumsgfmt... no
checking for gmsgfmt... no
checking for msgfmt... msgfmt
checking version of msgfmt... 0.19.8.1, ok
.....

Step 5: You might need the new version of GNU make to resolve the issue

To Download the make-4.2.1, do download at https://ftp.gnu.org/gnu/make/

To compile the make, it is very simple

# tar -zxvf make-4.2.1.tar.gz
# cd make-4.2.1
# ./configure --prefix=/usr/local/make-4.2.1
# make && make install

Step 6: Update the $PATH & $LD_LIBRARY_PATH

# export PATH=$PATH:/usr/local/make-4.2.1/bin
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/make-4.2.1/lib

Step 7: Repeat Step 3

GCCGO Error During GCC-10.4.0 Compilation on CentOS 7

If you encounter “gccgo: error: ../x86_64-pc-linux-gnu/libgo/libgotool.a: No such file or directory”

.....
.....
/home/user1/gcc-10.4.0/host-x86_64-pc-linux-gnu/gcc/gccgo -B/home/user1/gcc-10.4.0/host-x86_64-pc-linux-gnu/gcc/ -B/usr/x86_64-pc-linux-gnu/bin/ -B/usr/x86_64-pc-linux-gnu/lib/ -isystem /usr/x86_64-pc-linux-gnu/include -isystem /usr/x86_64-pc-linux-gnu/sys-include   -g -O2 -I ../x86_64-pc-linux-gnu/libgo -static-libstdc++ -static-libgcc  -L ../x86_64-pc-linux-gnu/libgo -L ../x86_64-pc-linux-gnu/libgo/.libs -o go ../.././gotools/../libgo/go/cmd/go/alldocs.go ../.././gotools/../libgo/go/cmd/go/go11.go ../.././gotools/../libgo/go/cmd/go/main.go ../x86_64-pc-linux-gnu/libgo/libgotool.a  
gccgo: error: ../x86_64-pc-linux-gnu/libgo/libgotool.a: No such file or directory
make[2]: *** [Makefile:821: go] Error 1
make[2]: Leaving directory '/home/user1/gcc-10.4.0/host-x86_64-pc-linux-gnu/gotools'
make[1]: *** [Makefile:14649: all-gotools] Error 2
make[1]: Leaving directory '/home/user1/gcc-10.4.0'
make: *** [Makefile:997: all] Error 2

The issue can be easily resolved by not building gcc in the same directory as the source code. At GCC Home

% ./contrib/download_prerequisites
% mkdir build
% ../configure --prefix=/usr/local/gcc-10.4.0 --disable-multilib --enable-languages=all
% make -j 8
% make install

Compiling GCC 12.1.0 on Rocky Linux 8.5

Option 1: The longer and customised method

Step 1: Download the following prerequisites applications libraries from https://gcc.gnu.org/pub/gcc/infrastructure/

  1. gmp-6.2.1
  2. mpfr-4.1.0
  3. mpc-1.2.1

Step 1. Install gmp-6.2.1

% bunzip2 gmp-6.2.1.tar.bz2
% tar -xvf gmp-6.2.1.tar
% cd gmp-6.2.1
% ./configure --prefix=/usr/local/gmp-6.2.1
% make 
% make install

Step 2: Install mpfr-4.1.0 (requires gmp-6.2.1 as prerequisites)

% bunzip2 mpfr-4.1.0.tar.bz
% tar -xvf mpfr-4.1.0.tar
% cd mpfr-4.1.0/
% ./configure --prefix=/usr/local/mpfr-4.1.0 --with-gmp=/usr/local/gmp-6.2.1/
% make
% make install

Step 3: Install mpc-1.2.1 (requires gmp-6.2.1 and mpfr-4.1.0)

% tar -zxvf mpc-1.2.1.tar.gz
% cd mpc-1.2.1/
% ./configure --prefix=/usr/local/mpc-1.2.1 -with-gmp=/usr/local/gmp-6.2.1 --with-mpfr=/usr/local/mpfr-4.1.0
% make
% make install

Step 4: Install isl-0.24 (requires gmp-6.2.1 as prerequisites)

% bunzip2 isl-0.24.tar.bz2
% tar -xvf isl-0.24.tar
% cd isl-0.24
% ./configure --prefix=/usr/local/isl-0.24 --with-gmp-prefix=/usr/local/gmp-6.2.1/
% make
% make install

Configure and Build GCC

% git clone git://gcc.gnu.org/git/gcc.git
% cd gcc
% mkdir build-gcc
% cd build-gcc
% ../configure --prefix=/usr/local/gcc-12.1 --enable-bootstrap --enable-languages=c,c++,fortran,lto --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux --with-static-standard-libraries --with-gmp=/usr/local/gmp-6.2.1 --with-mpc=/usr/local/mpc-1.2.1 --with-mpfr=/usr/local/mpfr-4.1.0 --with-isl=/usr/local/isl-0.24 --with-isl-lib=/usr/local/isl-0.24/lib --with-isl-include=/usr/local/isl-0.24/include

You may encounter issues like

/usr/local/software/gcc/build-gcc/./gcc/cc1: error while loading shared libraries: libisl.so.23: cannot open shared object file: No such file or directory

An alternative way is to let GCC do the download for you….. Retracing the steps

% git clone git://gcc.gnu.org/git/gcc.git
% cd gcc
% contrib/download_prerequisites
% mkdir build-gcc
% cd build-gcc
% ../configure --prefix=/usr/local/gcc-12.1 --enable-bootstrap --enable-languages=c,c++,fortran,lto --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux --with-static-standard-libraries

If you encounter any errors during make such as

/usr/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory

It is due to missing glibc-devel and glibc-devel.i686. You have to do a dnf install glibc-devel and glibc-devel.i686

% dnf install glibc-devel glibc-devel.i686

Option 2: The Faster Method

You can take a look at Compiling GCC-10.4.0 on CentOS-7 and tune to GCC-12.1.0

References:

References:

https://gcc.gnu.org/wiki/InstallingGCC

Compiling OpenMPI-3.1.6 with GCC-6.5

We assumed that you have installed GNU 6.5 and isl-0.15

Download the latest OpenMPI 3.1.6 package from OpenMPI site

% ./configure --prefix=/usr/local/gnu/openmpi-3.1.6 --enable-orterun-prefix-by-default --enable-mpi-cxx --enable-openib-rdmacm-ibaddr --enable-mca-no-build=btl-uct

–enable-orterun-prefix-by-default (Configure OMPI –enable-orterun-prefix-by-default and so that you do not need to add the prefix option)
–enable-openib-rdmacm-ibaddr (To enable routing over IB)
–enable-mpi-cxx (C++ bindings are no more built by default)
–enable-mca-no-build=btl-uct (ecent OpenMPI versions contain a BTL component called ‘uct’, which can cause data corruption when enabled, due to conflict on malloc hooks between OPAL and UCM.)

% make all install | tee install.log

References:

  1. Intel Community – Caught Signal 11 (Segmentation Fault: Does not mapped to object at)
  2. Open MPI + Scalasca :Can not run mpirun command with option –prefix?

Using multiple LDFLAGS and CPPFLAGS

In very layman terms,

LDFLAGS refers for linker flags and is often user defined libraries
CPPLAGS is used by the preprocessor and is often the include directory

For example, if I’m compiling multiple LDFLAGS and CPPFLAGS which is required by guile-2.2.4

# ./configure --prefix=/usr/local/guile-2.2.4 LDFLAGS="-L/usr/local/libtool-2.4.6/lib -L/usr/local/gmp-6.1.0/lib" CPPFLAGS="-I/usr/local/libtool-2.4.6/include -I/usr/local/gmp-6.1.0/include"

Reverting back to CMake-3.9.6

When I was compiling CMAKE-3.11.4 with GNU-5.4.0, we encountered the error

"The C++ compiler does not support C++11 (e.g. std::unique_ptr)"

This was rather complex to solve. I believe if I upgrade my GNU Compilers, it might work. Somehow at GNU 5.4, it does not recognize the C++11 support.

But when I downgraded to Cmake-3.9.6, it works immediately without issues. Do look at https://cmake.org/files/v3.9/

# cd $CMAKE_HOME
#./bootstrap
# gmake

Dealing with The compiler /usr/bin/c++ has no C++11 support for CentOS 6

If you compiling using cmake and you make encounter error like this. This is despite the fact that you use the latest compiler like GNU-5.2.0. But somehow the cmake is still looking at the older gnu that comes with CentOS 6

[user1@node1 build]$ cmake ..
-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
CMake Error at CMakeLists.txt:24 (message):
The compiler /usr/bin/c++ has no C++11 support. Aborting.

To resolve this, you have to be more explicit in the cmake parameter point to the corrcet g++, gcc

[user1@node1 build]$ cmake .. -DCMAKE_CXX_COMPILER=/usr/local/gcc-5.2.0/bin/g++ -DCMAKE_C_COMPILER=/usr/local/gcc-5.2.0/bin/gcc