Relooking at usage of /home directory in Linux

File System in /home can be clustered very quickly nowadays where the amount of data are massive. Residing all the users in one /home directory will cause the administrator to quickly reach the limits of the File System especially if the File System is not a Parallel File System.

One thought is to perhaps using the /home directory to store their user specific files that govern the user environment such as .bashrc .bash_profile .login etc only. All other files such as output, project files etc should not be stored immediately under the /home/user1 files system. Of course no one should be running their computation on the home directory.

If the users is large and you know that the /home directory will be broken further into further hierachy which can be divided into more volume. For example, /home/student/users1 or /home/staff/user2, /home/Department/user3

For more information, do see http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/home.html

Basic Kickstart on CentOS 6 with DHCP-Less environment (Part 2)

cent-1

9. Put the CentOS CD into the Client and reboot. When you get to the initial CentOS 6 menu selection screen, you press “esc” so you will go to a boot prompt.

boot: linux ks=http://192.168.1.1/kickstart/base.cfg ksdevice=eth0 text asknetwork

(where 192.168.1.1 is the webserver which contains the anaconda file)

10. The System should install automatically and create an exact  copy with the same configuration and password

11. To ease modification of the anaconda file, add and remove packages you may want to install kickstart config files

# yum install system-config-kickstart

For more information, you can also read

Basic Kickstart on CentOS 6 with DHCP-Less environment (Part 1)

For this tutorial, we assume we wish to automate the installation of a CentOS Server using the Kickstart

Step 1: Setup of CentOS Server and looking at Anaconda Config File

  1. Just follow the typical setup of a Clean CentOS Server. Just insert the DVD in, follow the easy to  follow steps to install.
  2. Once you have completed, you will notice that the Anaconda, the Red Hat Installation system saves a copy of the installation setup config  file  that was created by your installation choices to /root/ks-anaconda.cfg
  3. You can use this file /root/ks-anaconda.cfg to create another identical machine
  4. You can also use it to restore a machine to the original OS and then port the backup on top back.

Step 2: Setting up a On Demand Kickstart Server

    1. Create a folder “kickstart” on the root of the web server (/var/www/html/kickstarts)
    2. Copy /root/ks-anaconda.cfg to /var/www/html/kickstarts
    3. Rename the ks-anaconda.cfg to base.cfg (or any name you wish)
    4. Change the permission so that it can be read by the web server
      # chmod 744 base.cfg
    5. Tweak the base.cfg
    6. Edit base.cfg….. Uncomment out the lines
      clearpart --linux --drives=sda
      part /boot --fstype ext3 --size=100 --ondisk=sda
      part pv.3 --size=0 --grow --ondisk=sda
      volgroup VolGroup00 --pesize=32768 pv.3
      logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
      logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=1000 --grow --maxsize=5952
    7. Edit base.cfg . If you are using Static IP Addresses
      network --device eth0 --bootproto static
      --ip 192.168.1.2 --netmask 255.255.255.0
      --gateway 192.168.1.1
      --nameserver 192.168.1.100,192.168.101
      --hostname mylinux.homelinux.org
    8. Change the Installation Method.
      On the 2nd line of base.cfg, change the original “cdrom” to the web install

      url --url=http://url-to-web-install-server/CentOS-6.8
      ....
      ....
      ....
      repo --name="CentOS" --baseurl="http://url-to-web-install-server/CentOS-6.8"
    9. If you wish to disable selinux especially if you are setting up a cluster
      selinux --disabled
    10. See Basic Kickstart on CentOS 5 (Part 2) for the the rest of the tutorial….

Compiling BLACS on CentOS 5

1. You have to compile OpenMPI 1.4.x with g77 and gfortran. I’m compiling with OpenIB and Torque as well

./configure --prefix=/usr/local/mpi/gnu-g77/ \
F77=g77 FC=gfortran \
--with-openib \
--with-openib-libdir=/usr/lib64 \
--with-tm=/opt/torque

2. Download BLACS from www.netlib.org/blacs. Remember to download both mpiblacs.tgz and the mpiblacs-patch03.tgz

# cd /root
# tar -xzvf mpiblacs.tgz
# tar -xzvf mpiblacs-patch03.tgz
# cd BLACS
# cp ./BMAKES/BMake.MPI-LINUX Bmake.inc

3. Edit Bmake.inc according to the recommendation from OpenMPI FAQ

# Section 1:
# Ensure to use MPI for the communication layer

   COMMLIB = MPI

# The MPIINCdir macro is used to link in mpif.h and
# must contain the location of Open MPI's mpif.h. 
# The MPILIBdir and MPILIB macros are irrelevant
# and should be left empty.

   MPIdir = /path/to/openmpi-1.4.3
   MPILIBdir =
   MPIINCdir = $(MPIdir)/include
   MPILIB =

# Section 2:
# Set these values:

   SYSINC =
   INTFACE = -Df77IsF2C
   SENDIS =
   BUFF =
   TRANSCOMM = -DUseMpi2
   WHATMPI =
   SYSERRORS =

# Section 3:
# You may need to specify the full path to
# mpif77 / mpicc if they aren't already in
# your path. IF not type the whole path out.

   F77            = /usr/local/mpi/gnu-g77/bin/mpif77
   F77LOADFLAGS   =

   CC             = /usr/local/mpi/gnu-g77/bin/mpicc
   CCLOADFLAGS    =

4. Following the recommendation from BlACS Errata (Necessary flags for compiling the BLACS tester with g77)

blacstest.o : blacstest.f
	$(F77) $(F77NO_OPTFLAGS) -c $*.f
to:
blacstest.o : blacstest.f
	$(F77) $(F77NO_OPTFLAGS) -fno-globals -fno-f90 -fugly-complex -w -c $*.f

5. Compile the Blacs tests. You should see

# cd /root/BLACS/TESTING
# make clean
# make

You should see xCbtest_MPI-LINUX-1 and xFbtest_MPI-LINUX-1

6. Tun the Tests

# mpirun -np 5 xCbtest_MPI-LINUX-0
# mpirun -np 5 xFbtest_MPI-LINUX-0

7. If the test is successful, you may wish to copy the BLACS library to /usr/local/lib. But I like to  separate my compiled libraries separately to /usr/local/blacs/lib

# cp /root/BLACS/LIB*.a /usr/local/blacs/lib
# chmod 555 /usr/local/blacs/lib/*.a

Compiling LAPACK on CentOS 5

Download the lapack latest stable version (lapack-3.3.0.tgz) from http://www.netlib.org/lapack

# cd /root
# tar -xzvf lapack-3.3.0.tgz
# cd /root/lapack-3.3.0
# cp make.inc.example make.inc

Assuming Edit make.inc. Assuming the Compiling ATLAS on CentOS 5

#BLASLIB = ../../blas$(PLAT).a
BLASLIB = /usr/local/atlas/lib/libf77blas.a /usr/local/atlas/lib/libatlas.a

Compile lapack package

# make

Copy the libraries to

# mkdir /usr/local/lapack/lib
# cp /root/lapack-3.3.0/*.a /usr/local/lapack/lib
# cd /usr/local/lapack/lib/
# chmod 555 *.a

Other related Information

  1. Compiling ATLAS on CentOS 5

Compiling ATLAS on CentOS 5

This tutorial is to help you compile ATLAS (Automatically Tuned Linear Algebra Software) with gFortran. For those who are using Intel Compiler, you have the reliable Intel MKL (Math Kernel Library)

First thing first, some comparison between ATLAS and MKL.

ATLAS

ATLAS The Automatically Tuned Linear Algebra Software (ATLAS) provides a complete implementation of the BLAS API 3 and a subset of LAPACK 3. A big number of instructions-set specific optimizations are used throughout the library to achieve peak-performance on a wide variety of HW-platforms.

ATLAS provides both C and Fortran interfaces.

ATLAS is available for all HW-platforms capable of running UNIX or UNIX-like operating systems as well as Windows ™.

MKL

Intel’s Math Kernel Library (MKL) implements a set of linear algebra, fast Fourier transforms and vector math functions. It includes LAPACK 3, BLAS 3 and extended BLAS and provides both C and Fortran interfaces.

MKL is available for Windows ™ and Linux (x86/i686 and above) only.

Download the latest stable package from ATLAS (http://sourceforge.net/projects/math-atlas/files/Stable/). The current stable version is atlas3.8.0.tar.gz. Do note that ATLAS don’t like configuration on its original location, hence the need to create ATLAS_BUILD directory.

# cd /root
# tar -xzvf atlas3.8.3.tar.gz
# mkdir /root/ATLAS_BUILD
# cd /root/ATLAS_BUILD
# /root/ATLAS/configure

You will need to turn off CPU Throttling. For CentOS and Fedora, you will use

# /usr/bin/cpufreq-selector -g performance

For more information, you can see my blog entry Switching off CPU Throttling on CentOS or Fedora

Compile ATLAS

make
make check
make ptcheck
make time
make install

By default, ATLAS installed to /usr/local/atlas

Finally remember to add /usr/local/atlas/lib to your LD_LIBRARY_PATH

Notes:

    1. Linux Cluster Application Site
    2. ScaLAPACK, LAPACK, BLACS and ATLAS on OpenMPI & Linux installation tutorial