Author: kittycool only

by

Dependencies for pillow module for python-3 on CentOS 6

If you do a pip install pillow for python-3 on CentOS 6, you may encounter a broken and exception error.

# pip install pillow
......
......
ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting

Solution: Install “libjpeg-turbo-devel” and “libjpeg-turbo-static” which is an important dependencies for pillow module

# yum install libjpeg*
# easy_install pillow
Searching for pillow
Reading https://pypi.python.org/simple/pillow/
Best match: Pillow 3.1.0
..........
..........
Installed /usr/local/python-3.4.3/lib/python3.4/site-packages/Pillow-3.1.0-py3.4-linux-x86_64.egg
Processing dependencies for pillow
Finished processing dependencies for pillow
by

Basic Configuration of Octopus 5.0.0 with OpenMPI on CentOS 6

Octopus

Octopus is a scientific program aimed at the ab initio virtual experimentation on a hopefully ever-increasing range of system types. Electrons are described quantum-mechanically within density-functional theory (DFT), in its time-dependent form (TDDFT) when doing simulations in time. Nuclei are described classically as point particles. Electron-nucleus interaction is described within the pseudopotential approximation.

Requirements: (Taken from Octopus Installation Wiki)

In a nutshell, this is what you need. Do look at Octopus Wiki for more details

  1. make
  2. cpp
  3. LibXC – Octopus 5.0.0 requires version 2.1.x, and won’t compile with 2.2.x (Keep getting Errors.
  4. FFTW
  5. LAPACK/BLAS
  6. GSL – Version 4.0 of Octopus (and earlier) can only use GSL 1.14 (and earlier). A few tests will fail if you use GSL 1.15 or later.
  7. Perl

Step 1: Compile libXC. You can download libxc.2.1.2 from Octopus
Compile libXC. After untaring, do take a look at the INSTALL

# tar -zxvf libxc-2.1.2
# cd libxc-2.1.2
# ./configure --prefix=/usr/local/libxc-2.1.2/ CC=gcc CXX=g++
# make - j 16
# make install

Step 2: Compile gsl-1.14
Do take a look at Compiling GNU Scientific Library (GSL) gsl-1.16 on CentOS 6. Do look at ftp://ftp.gnu.org/gnu/gsl/

Step 3: Update your .bashrc

.......
export FC=mpif90
export CC=mpicc
export FCFLAGS="-O3"
export CFLAGS="-O3"
export PATH=$PATH:/usr/local/openmpi-1.6.4-gnu/bin:...........
export LD_LIBRARY_PATH: $LD_LIBRARY_PATH: /usr/local/openmpi-1.6.4-gnu/lib:
/usr/local/fftw-3.3.3-single/lib:/usr/local/libxc-2.1.2/lib...................

 

Step 4: Configure the Octopus-4.1.2

# ./configure 
--prefix=/usr/local/octopus-5.0.0  \
--with-libxc-prefix=/usr/local/libxc-2.1.2 \
--with-libxc-include=/usr/local/libxc-2.1.2/include \
--with-gsl-prefix=/usr/local/gsl-1.16 \
--with-blas=/usr/lib64/libblas.a \ 
--with-arpack=/usr/lib64/libarpack.so.2 \ 
--with-fft-lib="-L/usr/local/fftw-3.3.3-single/lib" \
--disable-zdotc-test \
--enable-single \
--enable-mpi

# make -j 16
# make install

 

by

GPFS Client Node cannot be added to the GPFS cluster

At the NSD Node, I issue the command

# mmaddnode -N node1
Thu Jul 23 13:40:12 SGT 2015: mmaddnode: Processing node node1
mmaddnode: Node node1 was not added to the cluster.
The node appears to already belong to a GPFS cluster.
mmaddnode: mmaddnode quitting.  None of the specified nodes are valid.
mmaddnode: Command failed.  Examine previous error messages to determine cause.

If we do a mmcluster, the node is not around in the cluster

# mmcluster |grep node1

If the node is not in the cluster, issue this command on the client node that could not be added:

# mmdelnode -f
mmdelnode: All GPFS configuration files on node goldsvr1 have been removed.

Reissue the mmaddnode command. References:

  1. Node cannot be added to the GPFS cluster
by

Compiling GNU 5.2.0 on CentOS 6

1. Prerequisites:

You will need

  1. gmp 4.3.1-7.el6_2.2
  2. mpfr  2.4.1-6.el6
  3. mpc 0.19-1.el6.rf and
  4. isl-0.14

For 1 to 3, do take a look at Compiling GNU 4.8.1 on CentOS 6. For 4, do take look at Compiling isl-0.15 library

2a. Update linked library at /etc/ld.so.conf

# touch /etc/ld.so.conf.d/library.conf
/usr/local/isl-0.15/lib
/usr/local/mpfr-2.4.2/lib
/usr/local/mpc-0.8.1/lib
/usr/local/gmp-4.3.2/lib

2b. Update your .bashrc

export LD_LIBRARY_PATH=/usr/local/mpfr-2.4.2/lib:/usr/local/mpc-0.8.1/lib:/usr/local/gmp-4.3.2/lib:/usr/local/isl-0.15/lib

3. Compile. This will take a while.

# tar -zxvf  gcc-4.8.1.tar.gz
# cd gcc-4.8.1
# mkdir build-gcc
# cd build-gcc
# ../configure --prefix=/usr/local/gcc-5.2.0 
--with-mpfr=/usr/local/mpfr-2.4.2 
--with-mpc=/usr/local/mpc-0.8.1 
--with-gmp=/usr/local/gmp-4.3.2
--with-isl=/usr/local/isl-0.14
--disable-multilib
--disable-libmpx
--enable-shared
--enable-languages=c,c++,fortran,go,objc,obj-c++  
--host x86_64-redhat-linux-gnu 
--build x86_64-redhat-linux-gnu
# make -j 16
# make install

References:

  1. Encountering error -static-libstdc++ not implemented when compiling GNU 4.8.1 on CentOS 6
  2. Configure Error when compiling GCC 5.2.0 on CentOS 6.6
by

Using Python 2 on JupyterHub

By default, JupyterHub uses Python 3.3. However you may want to use Python-2 on JuypterHub. You may want to take a look at Basic Setup and Configuration of JupyterHub with Python-3.4.3

Step 1: Install latest version of Python-2

You may want to see Installing and Compiling Python 2.7.8 on CentOS 5. You can apply this for CentOS 6

Step 2: Remember to install iPython2 and iPython[notebook] on Python-2

Do take a look at Installing scipy and other scientific packages using pip3 for Python 3.4.1 for some similar ideas

 Step 3: Install Python KernelSpec for Python 2

# /usr/local/python-2.7.10/bin/python2 -m IPython kernelspec install-self
# /usr/local/python-3.4.3/bin/python3 -m IPython kernelspec install-self

Step 4: Restart JupytHub

# juypterHub
by

Basic Setup and Configuration of JupyterHub with Python-3.4.3

This is a basic setup and configuration of JupyterHub from

Prerequisites:

  • JupyterHub requires IPython >= 3.0 (current master) and Python >= 3.3
  • Need to install nodejs/npm

Step 1: Install Python-3.4.3
You can use the tutorial to learn how to setup Python 3 (Compiling and Configuring Python 3.4.1 on CentOS)

Step 2: Install Nodejs and npm and Javascript Dependencies. You will need to install and enable EPEL repository

# yum install nodejs npm
# npm install -g configurable-http-proxy

Step 3a: Installation of JupyerHub

# pip3 install "ipython[notebook]"
# pip3 install jupyterhub
# git clone https://github.com/jupyter/jupyterhub.git
# cd jupyterHub
# pip3 install -r dev-requirements.txt -e .

Step 3b: Update Javascript

# python3 setup.py js
# python3 setup.py css

Step 4a: Update .bashrc or /etc/profile.d for python-3 path if you wish to affect global settings

# export PATH=/usr/local/python-3.4.3/bin:$PATH

Step 4b: Launch the JupyterHub Server

# jupyterhub

and then visit `http://localhost:8000`, and sign in with your unix credentials. If it does not work, no worry, just read on

Step 5: Generate a default config file:

# jupyterhub --generate-config

Step 6: Create Group shadow and put users into the group

The intention is to allow users to read the /etc/shadow file which is a requirements for jupytehub

# groupadd shadow
# chown root.shadow /etc/shadow
# usermod -G shadow user1
# chmod 040 /etc/shadow

If you are using DNS name instead of localhost, you would have to modify the jupyterhub_config.py found in /usr/local/juypterhub.
At approximately line 46, 181, modify localhost to your public-facing IP Address for the c.JupyterHub.hub_ip

c.JupyterHub.hub_ip = '10.10.10.10'

Step 6: Using sudo to run JupyterHub without root privilege

Do read the important document from JupyterHub wiki Using sudo to run JupyterHub without root privilege

Launch jupyterhub again.

# jupyterhub

 

by

Node cannot be added to the GPFS cluster

At the NSD Node, I issue the command

# mmaddnode -N node1
Thu Jul 23 13:40:12 SGT 2015: mmaddnode: Processing node node1
mmaddnode: Node node1 was not added to the cluster.
The node appears to already belong to a GPFS cluster.
mmaddnode: mmaddnode quitting.  None of the specified nodes are valid.
mmaddnode: Command failed.  Examine previous error messages to determine cause.

If we do a mmcluster, the node is not around in the cluster

# mmcluster |grep node1

If the node is not in the cluster, issue this command on the client node that could not be added:

# mmdelnode -f
mmdelnode: All GPFS configuration files on node goldsvr1 have been removed.

Reissue the mmaddnode command.

References:

  1. Node cannot be added to the GPFS cluster
by

GPFS unable to mount with no Errors Symptoms

When a GPFS client machine rebooted, the GPFS File System was unmounted with no error sign. When NSD issues the command “mmstartup -N client_node“, similarly, there is no error sign.

But if you do a

# mmdsh -v -N all "/usr/lpp/mmfs/bin/mmfsadm dump waiters" > all.waiters

You may see something like

.......Sync handler: on ThCond 0x1015236D30 (0xFFFFC20015236D30) (wait for inodeFlushFlag), reason 'waiting for the flush flag'......

This occurs when a revoke comes in and an mmapped file needs to be flushed to disk. GPFS tells Linux to flush all dirty mapped pages, and the thread then waits for Linux to report that this has been completed. So something in the kernel is preventing all the dirty pages from being flushed. I guess the best way is to use the NSD nodes to issue a command to do a

# mmshutdown -N client_node
# mmstartup -N client_node

Refereces:

  1. GPFS will not mount (but shows no errors)