OS Prerequisites
- RedHat EL4
- RedHat EL5
- SuSE SLES 10
- SuSE SLES 11
- Cent OS 5
Software Prerequisites
- bash-3.x.x
- glibc-2.3.x.x
- libgcc-3.4.x-x
- libstdc++-3.4.x-x
- perl-5.8.x-x
- tcl 8.4
- tk 8.4.x-x
- rpm 4.1.x-x
- libgfortran 4.1.x-x
Step 1: Download the Voltaire Drivers that is fitting to your OS and version.
Do find the link for Voltaire QDR Drivers at Download Voltaire OFED Drivers for CentOS
Step 2: Unzip and Untar the Voltaire OFED Package
# bunzip2 VoltaireOFED-1.5_3-k2.6.18-164.el5-x86_64.tar.bz # tar -xvf VoltaireOFED-1.5_3-k2.6.18-164.el5-x86_64.tar
Step 3: Install the Voltaire OFED Package
# cd VoltaireOFED-1.5_3-k2.6.18-164.el5-x86_64 # ./install
Step 3a: Reboot the Server
Step 4: Setup ip-over-ib
# vim /etc/sysconfig/network-scripts/ifcfg-ib0
# Voltaire Infiniband IPoIB DEVICE=ib0 ONBOOT=yes BOOTPROTO=static IPADDR=10.10.10.1 NETWORK=10.10.10.0 NETMASK=255.255.255.0 BROADCAST=10.10.255.255 MTU=65520
# service openibd start
Step 5 (Optional): Disable yum repository.
If you plan to use yum to local install the opensmd from the Voltaire package directory, you can opt for disabling the yum.
# vim /etc/yum.conf
Type the following at /etc/yum.conf
enabled=0
Step 6: Install Subnet Manager (opensmd). This can be found under
# cd $VoltaireRootDirectory/VoltaireOFED-1.5_3-k2.6.18-164.el5-x86_64/x86_64/2.6.18-164.15.1.el5
Yum install the opensmd packages
# yum localinstall opensm* --nogpgcheck
Restart the opensmd service
# service opensmd start
Step 7: Check that the Infiniband is working
# ibstat
You should get “State: Active”
CA 'mlx4_0' CA type: MT26428 Number of ports: 1 Firmware version: 2.6.0 Hardware version: a0 Node GUID: 0x0008f1476328oaf0 System image GUID: 0x0008fd6478a5af3 Port 1: State: Active Physical state: LinkUp Rate: 40 Base lid: 2 LMC: 0 SM lid: 14 Capability mask: 0x0251086a Port GUID: 0x0008f103467a5af1
Step 8: Test Connectivity
At the Server side,
# ibping -S
Do Step 1 to 7 again for the Client. Once done,
# ibping -G 0x0008f103467a5af1 (PORT GUID)
You should see a response like this.
Pong from headnode.cluster.com.(none) (Lid 2): time 0.062 ms Pong from headnode.cluster.com.(none) (Lid 2): time 0.084 ms Pong from headnode.cluster.com.(none) (Lid 2): time 0.114 ms Pong from headnode.cluster.com.(none) (Lid 2): time 0.082 ms Pong from headnode.cluster.com.(none) (Lid 2): time 0.118 ms Pong from headnode.cluster.com.(none) (Lid 2): time 0.118 ms
Great! you are done.