The IPoIB driver supports two modes of operation: Unreliable Datagram (UD) and Connected Mode.

In Unreliable datagram mode, the IB UD (Unreliable Datagram) transport is used and so the interface MTU has is equal to the IB L2 MTU minus the IPoIB encapsulation header (4 bytes).  In QDR, the default NTU value is 2044. In FDR onwards, the default MTU value for Unreliable Datagram is 4096.

In Connected Mode, the IB RC (Reliable Connected) transport is used.Connected mode takes advantage of the connected nature of the IB transport and allows an MTU up to the maximal IP packet size of 64K, which reduces the number of IP packets needed for handling large UDP datagrams, TCP segments, etc and increases the performance for large messages. Default MTU will be 65000. Performance will be better

To verify what modes you are working on, just do a

# cat /sys/class/net/ib0/mode
Datagram