If you are using Mellanox IB Switches, you can use the following to do conduct performance tests, these are:

Latency Server Side:

1. ib_write_lat
2. ib_read_lat
3. ib_send_lat

Latency Client Side:

1. ib_write_lat IP_Addresses
2. ib_read_lat IP_Addresses
3. ib_send_lat IP_Addresses

For examples:

1a. Latency Server Side

# ib_read_lat

1b. Client Side

# ib_read_lat IP_Address_of_Server -F -a
#bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]
 2       1000          1.66           12.98        1.70
 4       1000          1.64           13.40        1.67
 8       1000          1.64           20.25        1.67
 16      1000          1.64           19.61        1.68
.....
.....
 4096    1000          2.94           18.45        2.99

2a. Bandwidth Server Side

# ib_read_bw

2b. Bandwidth Client Side

# ib_read_bw -F -a
#bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 2          1000             6.97               6.47               3.394435
....
....
8192       1000             5983.30            5982.07            0.765704
....
....
65536      1000             6075.37            6042.28            0.096676