Session Trunking for NFS available in RHEL-8

This Article is taken from Is NFS session trunking available in RHEL?

Session trunking, whereby one can have multiple TCP connections to the same NFS server with the same IP address is provided by nconnect. This feature is available in RHEL 8

RHSA-2020:4431 for the package(s) kernel-4.18.0-240.el8 or later.
RHBA-2020:4530 for the package(s) nfs-utils-2.3.3-35.el8 libnfsidmap-2.3.3-35.el8 or later.

You can configure on the client side up to 16 connection

[root@nfs-client ~]# mount -t nfs 192.168.0.2:/nfs-share /nfs-share -o nosharecache,nconnect=16

You can see by using the command

[root@nfs-client ~]# cat /proc/self/mountstats
......
......
RPC iostats version: 1.1  p/v: 100003/4 (nfs)
    xprt:   tcp 991 0 2 0 39 13 13 0 13 0 2 0 0
    xprt:   tcp 798 0 2 0 39 6 6 0 6 0 2 0 0
    xprt:   tcp 768 0 2 0 39 6 6 0 6 0 2 0 0
    xprt:   tcp 1013 0 2 0 39 4 4 0 4 0 2 0 0
    xprt:   tcp 828 0 2 0 39 4 4 0 4 0 2 0 0
    xprt:   tcp 702 0 2 0 39 2 2 0 2 0 2 0 0
    xprt:   tcp 783 0 2 0 39 2 2 0 2 0 2 0 0
    xprt:   tcp 858 0 2 0 39 2 2 0 2 0 2 0 0
.....

Someone recorded multiple performance increase when used on Pure Storage acting as a NFS Server. at Use nconnect to effortlessly increase NFS performance

Pix Taken from Use nconnect to effortlessly increase NFS performance

References:

  1. Is NFS session trunking available in RHEL?
  2. Use nconnect to effortlessly increase NFS performance
  3. Explanation of NFS mount options: nconnect=, nosharetransport, sharetransport=

How to configure NFS on CentOS 7

Step 1: Do a Yum Install

# yum install nfs-utils rpcbind

Step 2: Enable the Service at Boot Time

# systemctl enable nfs-server
# systemctl enable rpcbind
# systemctl enable nfs-lock     (it does not need to be enabled since rpc-statd.service  is static.)
# systemctl enable nfs-idmap    (it does not need to be enabled since nfs-idmapd.service is static.)

Step 3: Start the Services

# systemctl start rpcbind
# systemctl start nfs-server
# systemctl start nfs-lock
# systemctl start nfs-idmap

Step 4: Confirm the status of NFS

# systemctl status nfs

Step 5: Create a mount point

# mkdir /shared-data

Step 6: Exports the Share

# vim /etc/exports
/shared-data 192.168.0.0/16(rw,no_root_squash)

Step 7: Export the Share

# exportfs -rv

Step 8: Restart the NFS Services

# systemctl restart nfs-server

Step 9: Configure the Firewall

# firewall-cmd --add-service=nfs --zone=internal --permanent
# firewall-cmd --add-service=mountd --zone=internal --permanent
# firewall-cmd --add-service=rpc-bind --zone=internal --permanent

References:

  1. How to configure NFS in RHEL 7
  2. What firewalld services should be active on an NFS server in RHEL 7?

How to unmount NFS mount that fails to unmount with ‘device is busy’

If you are attempting to unmount a NFS command like

# mount -t nfs -o remount /mnt/nfs 
# umount /mnt/nfs 
# umount -f /mnt/nfs 
# umount -l /mnt/nfs 
# umount -lf /mnt/nfs

Identify which processes tied to the mount need to be killed by using lsof and fuser:

# lsof | grep /mnt/nfs

lsof command above identifies the PID of the processes associated with the /mnt/nfs share. Kill any processes locking the stale mount.

Try to force umount again after the processes as been killed

# umount -lf

References:

  1. How to unmount a stale NFS mount that fails to unmount with ‘device is busy’ after network disconnectivity?

How to increase the number of threads created by the NFS daemon for CENTOS 7

Taken from How to increase the number of threads created by the NFS daemon in RHEL 4, 5, 6 and 7?

In case of a NFS server with a high load, it may be advisable to increase the number of the threads created during the nfsd server start up.

Edit the following line in /etc/nfs.conf

% vim /etc/nfs.conf
#[nfsd]
# debug=0
threads=64
# host=
# port=0
# grace-time=90
# lease-time=90
# udp=y
# tcp=y

Testing whether it works….

% cat /proc/net/rpc/nfsd

According to the RH, “The first number is the total number of NFS server threads started. The second number indicates whether at any time all of the threads were running at once. The remaining numbers are a thread count time histogram.”

th 64 0 2.610 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

mount.nfs: requested NFS version or transport protocol is not supported

If you have encountered issues like

mount.nfs: requested NFS version or transport protocol is not supported

OR

mount.nfs4: Protocol not supported

To resolve this

Mount with NFS version 3 (with 4 verbose flags)

% mount -vvvv -t nfs -o vers=3 nfs-server:/share /mnt/nfs

References:

  1. Error “mount.nfs: requested NFS version or transport protocol is not supported” when attempting to mount an NFS share on Red Hat Enterprise Linux 6

showmount fails with clnt_create: RPC: Program not registered from a NFS client communicating with a NetApp filer

1. Assuming this is your mount command

mount -t nfs -o vers=3 XXX.XXX.XXX.XXX:/myserver/nfs /myclient/nfs

 

2. And if you are using showmount command from an NFS client and the following are observed

clnt_create: RPC: Program not registered

 

3. You have to access the NetApp Storage to check the NFS Protocol is enabled. I’m using the Netapp OnCommand System Manager

4. Check that the NFS Client can mount.

showmount --no-headers -e nfs_server
/ (everyone)

References:

  1. showmount fails with clnt_create: RPC: Program not registered when executed from a RHEL6 NFS client communicating with a NetApp filer

Using nfsstat to troubleshoot NFS performance issues

The write-up is taken from RedHat Using nfsstat and nfsiostat to troubleshoot NFS performance issues on Linux

NFS relies on the existing network infrastructure, any glitches on the network may affect the performance of the connection. One of the tools that can be used is nfsstat

% yum install nfs-utils

The nfsstat command

The nfsstat command displays statistical information about the NFS and Remote Procedure Call (RPC) interfaces to the kernel.

On Server Side,

% nfsstat -s
Server rpc stats:
calls badcalls badclnt badauth xdrcall
107310012 0 0 0 0

The most important field to check is the badcalls, which represents the total number of calls rejected by the RPC layer. When the badcalls is greater than 0, than the underlying network needs to be checked, as there might be latency.

 

On NFS Client Side,

% nfsstat -c
Client rpc stats:
calls retrans authrefrsh
23158 0 23172

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 7237 31% 7 0% 1443 6% 7874 34% 11 0%
read write create mkdir symlink mknod
578 2% 4548 19% 585 2% 1 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
0 0% 0 0% 0 0% 0 0% 0 0% 51 0%
fsstat fsinfo pathconf commit
25 0% 10 0% 5 0% 781 3%

The client is doing well as it has relatively few retransmission requests. If you are encountering excessive retransmissions, you may want to adjust data transfer buffer sizes, which are specified by the mount command options rsize and size.

 

Check for dropped packets

Check dropped packet by running the following command on both the server and the client:

% nfsstat -o net
Client packet stats:
packets udp tcp tcpconn
0 0 0 0

NFS mount errors with “clnt_create: RPC: Unknown host” for CentOS 6

When attempting to mount CentOS 6, my mount fails with

clnt_create: RPC: Unknown host

Diagnostic:

If we do a more thorough diagnostic, this is the issue

# showmount -e  
clnt_create: RPC: Unknown host  
# showmount -e localhost  
Export list for localhost:  
/export/my_data \*

Resolution:

Taken from Redhat Site

Implement forward and reverse lookups (A records and CNAME records) in DNS and have the system point towards the DNS servers. Implement for both IPv4 and IPv6. If unable to resolve DNS issues, change the /etc/hosts file from this:

Change from

::1          localhost localhost.localdomain localhost6 localhost6.localdomain6

To

::1          machine_hostname localhost localhost.localdomain localhost6 localhost6.localdomain6

Restart the NFS service and check on the showmount -e localhost and showmount -e and attempt to mount the share.

# service nfs restart  
# showmount -e localhost  
# showmount -e