AnsweredAssumed Answered

NFS Throttling

Question asked by thealy on May 7, 2013
Latest reply on Nov 20, 2013 by thealy
Running M3 / 2.1.1.17042.GA
While uploading files via NFS, transfer stopped. mfs.log show continual (22,753 in past 40 mins) output of:

<code>INFO  procqueue.h:96 x.x.254.227:9998 Throttling request 123 from 111.222.254.227:9998. (Where 111.222.254.227 is the address of the NFS server itself)
</code>

nfsserver.log showed debug-like output with no timestamp around the time this started:

<code>totalLen bad, 1195725856
\*\*\*\*\*ONC: total Len too long, 1195725856
\*\*\*\*\*ONC: total Len too long, 1212501072
</code>

Later, a bunch (hundreds) of these:

<code>013-05-07 11:41:27,9985 ERROR nfsserver[29836] fs/nfsd/fileops.cc:3522 111.222.254.254[0xf1521462] Write Response: Connection timed out, err=110, status=0 [nfsfh=0.4209104640.73170.93.163810] fs=130.199.254.227:5660
....
013-05-07 11:41:27,9985 ERROR nfsserver[29836] fs/nfsd/fileops.cc:3522 111.222.254.254[0xf1521462] Write Response: Connection timed out, err=110, status=0 [nfsfh=0.4209104640.73170.93.163810] fs=111.222.254.227:5660
...
2013-05-07 11:42:08,5046 ERROR nfsserver[29836] fs/nfsd/requesthandle.cc:725 0.0.0.0[0] cannot allocate more OncRpcContexts: [numDropped=9001] dropping connection from  nfsc=111.222.254.254:0
2013-05-07 11:42:09,6367 ERROR nfsserver[29836] fs/nfsd/requesthandle.cc:725 0.0.0.0[0] cannot allocate more OncRpcContexts: [numDropped=9101] dropping connection from  nfsc=111.222.254.254:0
....
Timing out request 28.123 sent to 111.222.254.227:5660
Other ips are: 130.199.254.227:5660 192.168.4.10:5660
2013-05-07 11:41:32,6080 xmit to 111.222.254.227:5660 for 1 28.123 took 249815 msec
2013-05-07 11:41:32,6083 xmit to 111.222.254.227:5660 for 1 28.123 took 249731 msec
</code>

It is worth mentioning that the NFS server has two network interfaces.

I tried restarting NFS but that did not seem to help.


Outcomes