When attempting to mount CentOS 6, my mount fails with

clnt_create: RPC: Unknown host

Diagnostic:

If we do a more thorough diagnostic, this is the issue

# showmount -e  
clnt_create: RPC: Unknown host  
# showmount -e localhost  
Export list for localhost:  
/export/my_data \*

Resolution:

Taken from Redhat Site

Implement forward and reverse lookups (A records and CNAME records) in DNS and have the system point towards the DNS servers. Implement for both IPv4 and IPv6. If unable to resolve DNS issues, change the /etc/hosts file from this:

Change from

::1          localhost localhost.localdomain localhost6 localhost6.localdomain6

To

::1          machine_hostname localhost localhost.localdomain localhost6 localhost6.localdomain6

Restart the NFS service and check on the showmount -e localhost and showmount -e and attempt to mount the share.

# service nfs restart  
# showmount -e localhost  
# showmount -e

For more information on NFS4 and difference between NFS3 and NFS4, do look at A brief look at the difference between NFSv3 and NFSv4.

This tutorial is a guide on how to install NFSv4 on CentOS 5 and 6

Step1: Installing the packages

# yum install nfs-utils nfs4-acl-tools portmap

Some facts about the tools above as given from yum info.

nfs-utils –  The nfs-utils package provides a daemon for the kernel NFS server and related tools, which provides a much higher level of performance than the traditional Linux NFS server used by most users.

This package also contains the showmount program.  Showmount queries the mount daemon on a remote host for information about the NFS (Network File System) server on the remote host. For example, showmount can display the clients which are mounted on that host. This package also contains the mount.nfs and umount.nfs program.

nfs4-acl-tools – This package contains commandline and GUI ACL utilities for the Linux NFSv4 client.

portmap – The portmapper program is a security tool which prevents theft of NIS (YP), NFS and other sensitive information via the portmapper. A portmapper manages RPC connections, which are used by protocols like NFS and NIS.

The portmap package should be installed on any machine which acts as a server for protocols using RPC.

Step 2: Exports the File System from the NFS Server (Similar to NFSv3 except with the inclusion of fsid=0)

/home           192.168.1.0/24(rw,no_root_squash,sync,no_subtree_check,fsid=0)
/install        192.168.1.0/24(rw,no_root_squash,sync,no_subtree_check,fsid=1)

The fsid=0 and fsid=1 option provides a number to use in identifying the filesystem. This number must be different for all the filesystems in /etc/exports that use the fsid option. This option is only necessary for exporting filesystems that reside on a block device with a minor number above 255.one directory can be exported with each fsid option.

Exports the file system

# exportfs -av

Restart the NFS service

# service nfs start

If you are supporting NFSv3,  you have to start portmap as NFSv3 requires them. As such, NFSv4 does not need to interact with rpcbind[1], rpc.lockd, and rpc.statd daemons. For more information see Fedora Chapter 9.  Network File System (NFS) – How it works for a more in-depth understanding.

# service portmap restart

Step 2: Client Mapping

# mount -t nfs4 192.168.1.1:/ /home

For other information,

1. NFS4 Client unable to mount Server NFS4 file
2. A brief look at the difference between NFSv3 and NFSv4

There are a few interesting differences between NFSv3 and NFSv4. Comparison of  NFSv3 and NFSv4 is quite hard to obtain and the information is referenced from NFS Version 4 Open Source Project.

From a File System perspective, there are

Export Management

1. In NFSv3, client must rely on auxiliary protocol, the mount protocol to request a list of server’s exports and obtain root filehandle of a given export. It is fed into the NFS protocol proper once the root filehandle is obtained.
2. In NFSv4 uses the virtual file system to present the server’s export and associated root filehandles to the client.
3. NFSv4 defines a special operation to retrieve the Root filehandle and the NFS Server presents the appearance to the client that each export is just a directory in the pseudofs
4. NFSv4 Pseudo File System is supposed to provide maximum flexibility. Exports Pathname on servers can be changed transparently to clients.

State

1. NFSv3 is stateless. In other words if the server reboots, the clients can pick up where it left off. No state has been lost.
2. NFSv3 is typically used with NLM, an auxiliary protocol for file locking. NLM is stateful that the server LOCKD keeps track of locks.
3. In NFSv4, locking operations are part of the protocol
4. NFSv4 servers keep track of open files and delegations

Blocking Locks

1. NFSv3 rely on NLM. Basically, Client process is put to “sleep”. When a callback is received from the server, client process is granted the lock.
2. For NFSv4, the client to put to sleep, but will poll the server periodically for the lock.
3. The benefits of the mechanism is that there is one-way reachability from client to server. But it may be less efficient.

This article “Network File System ( NFS ) in High Performance Networks” by Carnegic Mellon is very interesting article about NFS Performance. Do take a look. Here is a summary of their fundings

1. For point-to-point throughput, IP over InfiniBand (Connected Mode) is comparable to a native InfiniBand.
2. When a disk is a bottleneck, NFS can benefit from neither IPoIB nor RMDA
3. When a disk is not a bottleneck, NFS benefits significantly from both IPoIB and RDMA. RDMA is better than IPoIB by ~20%
4. As the number of concurrent read operations increases, aggregate throughputs achieved for both IPoIB and RDMA significantly improve with no disadvantage for IPoIB

How to diagnose issues at NFS is definitely very important need-to-have knowledge for system administrator. Here are a few pointers to help system adminstrators along.

1. Testing for Saturated Network

I’ve taken most of this information from the article “NFS for Clusters” and “Linux NFS and Automounter Administration” by Erez Zadok

Profiling Write Operation at NFS

$ time dd if=/dev/zero of=testfile bs=4k count=16384
16384+0 records in
16384+0 records out
67108864 bytes (67 MB) copied, 0.518172 s, 130 MB/s
real    0m0.529s
user    0m0.016s
sys    0m0.500s

time = time a simple command or give resource usage
dd = convert and copy a file
if =  read from FILE instead of stdin
of =  write to FILE instead of stdin
bs = read and write BYTES bytes at a time
count = BLOCKS

According to Wikipedia /dev/zero is a special file that provides as many null characters (ASCII NUL, 0x00) as are read from it. One of the typical uses is to provide a character stream for overwriting information. Another might be to generate a clean file of a certain size. Like /dev/null, /dev/zero acts as a source and sink for data. All writes to /dev/zero succeed with no other effects (the same as for /dev/null, although /dev/null is the more commonly used data sink); all reads on /dev/zero return as many NULs as characters requested.
 

Profiling Read Operation for NFS

When profiling reads instead of writes, call umount and mount to flush caches, or the read might be instantaneous and give the impression of quick read

$ cd /
$ umount /mnt/shareddrive
$ mount /mnt/shareddrive
$ cd /mnt/shareddrive
$ dd if=testfile of=/dev/null bs=4k count=16384

Here after unmounting and mounting again the shared NFS, the testfile which exists on the shared drive is read and writen to /dev/null.

According to the article “NFS for Clusters“, if more than 3% of calls are retransmitted, then there are problems with the network or NFS server.
Look for NFS failures on a shared disk server with

$ nfsstat -s
or
$ nfsstat -o rpc