by

Installing Berkeley UPC on CentOS 6 and GNU-4.8

Preparation

  1. Download the Berkeley UPC from the Download Site. I use the http://upc.lbl.gov/download/
  2. Compile GNU 4.8.x. Do take a look at Compiling GNU 4.8.1 on CentOS 6
  3. Compile OpenMPI on GNU-4.8.1

Compilation

Step 1: Make sure your PATH and LD_LIBRARY_PATH reflect the GNU and OpenMPI Path

Step 2: Untar and Compile Berkeley UPC

# tar -zxvf berkeley_upc-2.22.0.tar.gz
# cd berkeley_upc-2.22.0
# mkdir build
# cd build
# ../configure CC=gcc CXX=g++ MPI_CC=mpicc --prefix=/usr/local/berkeley_upc-2.22.0
# gmake

Step 3: Untar and Compile the local UPC-C translator

Download the Berkeley Translator from the Download site. http://upc.lbl.gov/download/source.shtml#translator

# tar -zxvf berkeley_upc_translator-2.18.0.tar.gz
# cd berkeley_upc_translator-2.18.0
# make

After completion of successful build, it will show you the translator location path. Copy the line

translator = /usr/local/berkeley_upc_translator-2.18.0/open64/osprey1.0/build_ia64

and replace the line at the following files

/usr/local/RH6_apps/berkeley_upc-2.22.0/dbg/etc/upcc.conf
/usr/local/RH6_apps/berkeley_upc-2.22.0/opt/etc/upcc.conf

Step 4: Finally make install Berkeley UPC

# make install

Step 5: Test

# upcc --version
# upcrun

References:

  1. Installing Berkeley UPC on a Linux cluster
by

Dependencies for pillow module for python-3 on CentOS 6

If you do a pip install pillow for python-3 on CentOS 6, you may encounter a broken and exception error.

# pip install pillow
......
......
ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting

Solution: Install “libjpeg-turbo-devel” and “libjpeg-turbo-static” which is an important dependencies for pillow module

# yum install libjpeg*
# easy_install pillow
Searching for pillow
Reading https://pypi.python.org/simple/pillow/
Best match: Pillow 3.1.0
..........
..........
Installed /usr/local/python-3.4.3/lib/python3.4/site-packages/Pillow-3.1.0-py3.4-linux-x86_64.egg
Processing dependencies for pillow
Finished processing dependencies for pillow
by

Basic Configuration of Octopus 5.0.0 with OpenMPI on CentOS 6

Octopus

Octopus is a scientific program aimed at the ab initio virtual experimentation on a hopefully ever-increasing range of system types. Electrons are described quantum-mechanically within density-functional theory (DFT), in its time-dependent form (TDDFT) when doing simulations in time. Nuclei are described classically as point particles. Electron-nucleus interaction is described within the pseudopotential approximation.

Requirements: (Taken from Octopus Installation Wiki)

In a nutshell, this is what you need. Do look at Octopus Wiki for more details

  1. make
  2. cpp
  3. LibXC – Octopus 5.0.0 requires version 2.1.x, and won’t compile with 2.2.x (Keep getting Errors.
  4. FFTW
  5. LAPACK/BLAS
  6. GSL – Version 4.0 of Octopus (and earlier) can only use GSL 1.14 (and earlier). A few tests will fail if you use GSL 1.15 or later.
  7. Perl

Step 1: Compile libXC. You can download libxc.2.1.2 from Octopus
Compile libXC. After untaring, do take a look at the INSTALL

# tar -zxvf libxc-2.1.2
# cd libxc-2.1.2
# ./configure --prefix=/usr/local/libxc-2.1.2/ CC=gcc CXX=g++
# make - j 16
# make install

Step 2: Compile gsl-1.14
Do take a look at Compiling GNU Scientific Library (GSL) gsl-1.16 on CentOS 6. Do look at ftp://ftp.gnu.org/gnu/gsl/

Step 3: Update your .bashrc

.......
export FC=mpif90
export CC=mpicc
export FCFLAGS="-O3"
export CFLAGS="-O3"
export PATH=$PATH:/usr/local/openmpi-1.6.4-gnu/bin:...........
export LD_LIBRARY_PATH: $LD_LIBRARY_PATH: /usr/local/openmpi-1.6.4-gnu/lib:
/usr/local/fftw-3.3.3-single/lib:/usr/local/libxc-2.1.2/lib...................

 

Step 4: Configure the Octopus-4.1.2

# ./configure 
--prefix=/usr/local/octopus-5.0.0  \
--with-libxc-prefix=/usr/local/libxc-2.1.2 \
--with-libxc-include=/usr/local/libxc-2.1.2/include \
--with-gsl-prefix=/usr/local/gsl-1.16 \
--with-blas=/usr/lib64/libblas.a \ 
--with-arpack=/usr/lib64/libarpack.so.2 \ 
--with-fft-lib="-L/usr/local/fftw-3.3.3-single/lib" \
--disable-zdotc-test \
--enable-single \
--enable-mpi

# make -j 16
# make install

 

by

GPFS Client Node cannot be added to the GPFS cluster

At the NSD Node, I issue the command

# mmaddnode -N node1
Thu Jul 23 13:40:12 SGT 2015: mmaddnode: Processing node node1
mmaddnode: Node node1 was not added to the cluster.
The node appears to already belong to a GPFS cluster.
mmaddnode: mmaddnode quitting.  None of the specified nodes are valid.
mmaddnode: Command failed.  Examine previous error messages to determine cause.

If we do a mmcluster, the node is not around in the cluster

# mmcluster |grep node1

If the node is not in the cluster, issue this command on the client node that could not be added:

# mmdelnode -f
mmdelnode: All GPFS configuration files on node goldsvr1 have been removed.

Reissue the mmaddnode command. References:

  1. Node cannot be added to the GPFS cluster
by

Compiling GNU 5.2.0 on CentOS 6

1. Prerequisites:

You will need

  1. gmp 4.3.1-7.el6_2.2
  2. mpfr  2.4.1-6.el6
  3. mpc 0.19-1.el6.rf and
  4. isl-0.14

For 1 to 3, do take a look at Compiling GNU 4.8.1 on CentOS 6. For 4, do take look at Compiling isl-0.15 library

2a. Update linked library at /etc/ld.so.conf

# touch /etc/ld.so.conf.d/library.conf
/usr/local/isl-0.15/lib
/usr/local/mpfr-2.4.2/lib
/usr/local/mpc-0.8.1/lib
/usr/local/gmp-4.3.2/lib

2b. Update your .bashrc

export LD_LIBRARY_PATH=/usr/local/mpfr-2.4.2/lib:/usr/local/mpc-0.8.1/lib:/usr/local/gmp-4.3.2/lib:/usr/local/isl-0.15/lib

3. Compile. This will take a while.

# tar -zxvf  gcc-4.8.1.tar.gz
# cd gcc-4.8.1
# mkdir build-gcc
# cd build-gcc
# ../configure --prefix=/usr/local/gcc-5.2.0 
--with-mpfr=/usr/local/mpfr-2.4.2 
--with-mpc=/usr/local/mpc-0.8.1 
--with-gmp=/usr/local/gmp-4.3.2
--with-isl=/usr/local/isl-0.14
--disable-multilib
--disable-libmpx
--enable-shared
--enable-languages=c,c++,fortran,go,objc,obj-c++  
--host x86_64-redhat-linux-gnu 
--build x86_64-redhat-linux-gnu
# make -j 16
# make install

References:

  1. Encountering error -static-libstdc++ not implemented when compiling GNU 4.8.1 on CentOS 6
  2. Configure Error when compiling GCC 5.2.0 on CentOS 6.6
by

Using Python 2 on JupyterHub

By default, JupyterHub uses Python 3.3. However you may want to use Python-2 on JuypterHub. You may want to take a look at Basic Setup and Configuration of JupyterHub with Python-3.4.3

Step 1: Install latest version of Python-2

You may want to see Installing and Compiling Python 2.7.8 on CentOS 5. You can apply this for CentOS 6

Step 2: Remember to install iPython2 and iPython[notebook] on Python-2

Do take a look at Installing scipy and other scientific packages using pip3 for Python 3.4.1 for some similar ideas

 Step 3: Install Python KernelSpec for Python 2

# /usr/local/python-2.7.10/bin/python2 -m IPython kernelspec install-self
# /usr/local/python-3.4.3/bin/python3 -m IPython kernelspec install-self

Step 4: Restart JupytHub

# juypterHub
by

Basic Setup and Configuration of JupyterHub with Python-3.4.3

This is a basic setup and configuration of JupyterHub from

Prerequisites:

  • JupyterHub requires IPython >= 3.0 (current master) and Python >= 3.3
  • Need to install nodejs/npm

Step 1: Install Python-3.4.3
You can use the tutorial to learn how to setup Python 3 (Compiling and Configuring Python 3.4.1 on CentOS)

Step 2: Install Nodejs and npm and Javascript Dependencies. You will need to install and enable EPEL repository

# yum install nodejs npm
# npm install -g configurable-http-proxy

Step 3a: Installation of JupyerHub

# pip3 install "ipython[notebook]"
# pip3 install jupyterhub
# git clone https://github.com/jupyter/jupyterhub.git
# cd jupyterHub
# pip3 install -r dev-requirements.txt -e .

Step 3b: Update Javascript

# python3 setup.py js
# python3 setup.py css

Step 4a: Update .bashrc or /etc/profile.d for python-3 path if you wish to affect global settings

# export PATH=/usr/local/python-3.4.3/bin:$PATH

Step 4b: Launch the JupyterHub Server

# jupyterhub

and then visit `http://localhost:8000`, and sign in with your unix credentials. If it does not work, no worry, just read on

Step 5: Generate a default config file:

# jupyterhub --generate-config

Step 6: Create Group shadow and put users into the group

The intention is to allow users to read the /etc/shadow file which is a requirements for jupytehub

# groupadd shadow
# chown root.shadow /etc/shadow
# usermod -G shadow user1
# chmod 040 /etc/shadow

If you are using DNS name instead of localhost, you would have to modify the jupyterhub_config.py found in /usr/local/juypterhub.
At approximately line 46, 181, modify localhost to your public-facing IP Address for the c.JupyterHub.hub_ip

c.JupyterHub.hub_ip = '10.10.10.10'

Step 6: Using sudo to run JupyterHub without root privilege

Do read the important document from JupyterHub wiki Using sudo to run JupyterHub without root privilege

Launch jupyterhub again.

# jupyterhub

 

by

Node cannot be added to the GPFS cluster

At the NSD Node, I issue the command

# mmaddnode -N node1
Thu Jul 23 13:40:12 SGT 2015: mmaddnode: Processing node node1
mmaddnode: Node node1 was not added to the cluster.
The node appears to already belong to a GPFS cluster.
mmaddnode: mmaddnode quitting.  None of the specified nodes are valid.
mmaddnode: Command failed.  Examine previous error messages to determine cause.

If we do a mmcluster, the node is not around in the cluster

# mmcluster |grep node1

If the node is not in the cluster, issue this command on the client node that could not be added:

# mmdelnode -f
mmdelnode: All GPFS configuration files on node goldsvr1 have been removed.

Reissue the mmaddnode command.

References:

  1. Node cannot be added to the GPFS cluster