Suppose you have source and destination server with a IB Switch in-between
Server 1 is as below. I have changed the GUID and mask for confidentiality sake
[root@mtlacad03 ~]# ibstat CA 'mlx4_0' CA type: MT4103 Number of ports: 2 Firmware version: 2.33.5100 Hardware version: 0 Node GUID: 00000000000000 System image GUID: 00000000000 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 15 LMC: 0 SM lid: 5 Capability mask: 0000000000000 Port GUID: 00000000000 Link layer: InfiniBand Port 2: State: Down Physical state: Disabled Rate: 40 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 111111111111 Port GUID: 1111111111111 Link layer: Ethernet
Server 2 is
[root@mtlacad07 ~]# ibstat CA 'mlx4_0' CA type: MT4103 Number of ports: 2 Firmware version: 2.33.5100 Hardware version: 0 Node GUID: 000000000000 System image GUID: 0000000000 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 13 LMC: 0 SM lid: 5 Capability mask: 00000000000 Port GUID: 000000000000000 Link layer: InfiniBand Port 2: State: Down Physical state: Disabled Rate: 40 Base lid: 0 LMC: 0 SM lid: 0 Capability mask: 1111111111111 Port GUID: 1111111111111 Link layer: Ethernet
So if you do a ibtracert
[root@mtlacad07 ~]# ibtracert 13 15
From ca {0000000000000} portnum 1 lid 13-13 "mtlacad07 HCA-1"
[1] -> switch port {0000000000000}[2] lid 20-20 "MF0;Left-Leaf-SW03:SX6036G/U1"
[3] -> ca port {00000000000}[1] lid 15-15 "mtlacad03 HCA-1"
To ca {0000000000000} portnum 1 lid 15-15 "mtlacad03 HCA-1"
Basically, what it mean is that the
- Traffic is leaving mtlacad07 HCA-1 Port [1]
- Traffic is entering Switch Port 2 at the Left-Leaf Switch
- Traffic is leaving Switch Port 3 at the Left-Leaf Switch
- Traffic is entering mtlacad03 HCA-1 Port [1-
