File System in /home can be clustered very quickly nowadays where the amount of data are massive. Residing all the users in one /home directory will cause the administrator to quickly reach the limits of the File System especially if the File System is not a Parallel File System.
One thought is to perhaps using the /home directory to store their user specific files that govern the user environment such as .bashrc .bash_profile .login etc only. All other files such as output, project files etc should not be stored immediately under the /home/user1 files system. Of course no one should be running their computation on the home directory.
If the users is large and you know that the /home directory will be broken further into further hierachy which can be divided into more volume. For example, /home/student/users1 or /home/staff/user2, /home/Department/user3
For more information, do see http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/home.html
9. Put the CentOS CD into the Client and reboot. When you get to the initial CentOS 6 menu selection screen, you press “esc” so you will go to a boot prompt.
boot: linux ks=http://192.168.1.1/kickstart/base.cfg ksdevice=eth0 text asknetwork
(where 192.168.1.1 is the webserver which contains the anaconda file)
10. The System should install automatically and create an exact copy with the same configuration and password
11. To ease modification of the anaconda file, add and remove packages you may want to install kickstart config files
# yum install system-config-kickstart
For more information, you can also read
For this tutorial, we assume we wish to automate the installation of a CentOS Server using the Kickstart
Step 1: Setup of CentOS Server and looking at Anaconda Config File
- Just follow the typical setup of a Clean CentOS Server. Just insert the DVD in, follow the easy to follow steps to install.
- Once you have completed, you will notice that the Anaconda, the Red Hat Installation system saves a copy of the installation setup config file that was created by your installation choices to /root/ks-anaconda.cfg
- You can use this file /root/ks-anaconda.cfg to create another identical machine
- You can also use it to restore a machine to the original OS and then port the backup on top back.
Step 2: Setting up a On Demand Kickstart Server
- Create a folder “kickstart” on the root of the web server (/var/www/html/kickstarts)
- Copy /root/ks-anaconda.cfg to /var/www/html/kickstarts
- Rename the ks-anaconda.cfg to base.cfg (or any name you wish)
- Change the permission so that it can be read by the web server
# chmod 744 base.cfg
- Tweak the base.cfg
- Edit base.cfg….. Uncomment out the lines
clearpart --linux --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part pv.3 --size=0 --grow --ondisk=sda
volgroup VolGroup00 --pesize=32768 pv.3
logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=1000 --grow --maxsize=5952
- Edit base.cfg . If you are using Static IP Addresses
network --device eth0 --bootproto static
--ip 192.168.1.2 --netmask 255.255.255.0
- Change the Installation Method.
On the 2nd line of base.cfg, change the original “cdrom” to the web install
repo --name="CentOS" --baseurl="http://url-to-web-install-server/CentOS-6.8"
- If you wish to disable selinux especially if you are setting up a cluster
- See Basic Kickstart on CentOS 5 (Part 2) for the the rest of the tutorial….
1. You have to compile OpenMPI 1.4.x with g77 and gfortran. I’m compiling with OpenIB and Torque as well
./configure --prefix=/usr/local/mpi/gnu-g77/ \
F77=g77 FC=gfortran \
2. Download BLACS from www.netlib.org/blacs. Remember to download both mpiblacs.tgz and the mpiblacs-patch03.tgz
# cd /root
# tar -xzvf mpiblacs.tgz
# tar -xzvf mpiblacs-patch03.tgz
# cd BLACS
# cp ./BMAKES/BMake.MPI-LINUX Bmake.inc
3. Edit Bmake.inc according to the recommendation from OpenMPI FAQ
# Section 1:
# Ensure to use MPI for the communication layer
COMMLIB = MPI
# The MPIINCdir macro is used to link in mpif.h and
# must contain the location of Open MPI's mpif.h.
# The MPILIBdir and MPILIB macros are irrelevant
# and should be left empty.
MPIdir = /path/to/openmpi-1.4.3
MPIINCdir = $(MPIdir)/include
# Section 2:
# Set these values:
INTFACE = -Df77IsF2C
TRANSCOMM = -DUseMpi2
# Section 3:
# You may need to specify the full path to
# mpif77 / mpicc if they aren't already in
# your path. IF not type the whole path out.
F77 = /usr/local/mpi/gnu-g77/bin/mpif77
CC = /usr/local/mpi/gnu-g77/bin/mpicc
4. Following the recommendation from BlACS Errata (Necessary flags for compiling the BLACS tester with g77)
blacstest.o : blacstest.f
$(F77) $(F77NO_OPTFLAGS) -c $*.f
blacstest.o : blacstest.f
$(F77) $(F77NO_OPTFLAGS) -fno-globals -fno-f90 -fugly-complex -w -c $*.f
5. Compile the Blacs tests. You should see
# cd /root/BLACS/TESTING
# make clean
You should see xCbtest_MPI-LINUX-1 and xFbtest_MPI-LINUX-1
6. Tun the Tests
# mpirun -np 5 xCbtest_MPI-LINUX-0
# mpirun -np 5 xFbtest_MPI-LINUX-0
7. If the test is successful, you may wish to copy the BLACS library to /usr/local/lib. But I like to separate my compiled libraries separately to /usr/local/blacs/lib
# cp /root/BLACS/LIB*.a /usr/local/blacs/lib
# chmod 555 /usr/local/blacs/lib/*.a
Download the lapack latest stable version (lapack-3.3.0.tgz) from http://www.netlib.org/lapack
# cd /root
# tar -xzvf lapack-3.3.0.tgz
# cd /root/lapack-3.3.0
# cp make.inc.example make.inc
Assuming Edit make.inc. Assuming the Compiling ATLAS on CentOS 5
#BLASLIB = ../../blas$(PLAT).a
BLASLIB = /usr/local/atlas/lib/libf77blas.a /usr/local/atlas/lib/libatlas.a
Compile lapack package
Copy the libraries to
# mkdir /usr/local/lapack/lib
# cp /root/lapack-3.3.0/*.a /usr/local/lapack/lib
# cd /usr/local/lapack/lib/
# chmod 555 *.a
Other related Information
- Compiling ATLAS on CentOS 5
This tutorial is to help you compile ATLAS (Automatically Tuned Linear Algebra Software) with gFortran. For those who are using Intel Compiler, you have the reliable Intel MKL (Math Kernel Library)
First thing first, some comparison between ATLAS and MKL.
ATLAS The Automatically Tuned Linear Algebra Software (ATLAS) provides a complete implementation of the BLAS API 3 and a subset of LAPACK 3. A big number of instructions-set specific optimizations are used throughout the library to achieve peak-performance on a wide variety of HW-platforms.
ATLAS provides both C and Fortran interfaces.
ATLAS is available for all HW-platforms capable of running UNIX or UNIX-like operating systems as well as Windows ™.
Intel’s Math Kernel Library (MKL) implements a set of linear algebra, fast Fourier transforms and vector math functions. It includes LAPACK 3, BLAS 3 and extended BLAS and provides both C and Fortran interfaces.
MKL is available for Windows ™ and Linux (x86/i686 and above) only.
Download the latest stable package from ATLAS (http://sourceforge.net/projects/math-atlas/files/Stable/). The current stable version is atlas3.8.0.tar.gz. Do note that ATLAS don’t like configuration on its original location, hence the need to create ATLAS_BUILD directory.
# cd /root
# tar -xzvf atlas3.8.3.tar.gz
# mkdir /root/ATLAS_BUILD
# cd /root/ATLAS_BUILD
You will need to turn off CPU Throttling. For CentOS and Fedora, you will use
# /usr/bin/cpufreq-selector -g performance
For more information, you can see my blog entry Switching off CPU Throttling on CentOS or Fedora
By default, ATLAS installed to /usr/local/atlas
Finally remember to add /usr/local/atlas/lib to your LD_LIBRARY_PATH
- Linux Cluster Application Site
- ScaLAPACK, LAPACK, BLACS and ATLAS on OpenMPI & Linux installation tutorial