AnsweredAssumed Answered

Rmdirs failed , removeFid failed

Question asked by dhariaekta on Nov 4, 2014
Latest reply on Nov 21, 2016 by dhariaekta
Reducer does not work it stops at a certain percentage. Mapper gets completed.

When i type the following command (hadoop fs -ls /var/mapr/local/mapred/taskTracker) i get the output without

-rwxr-xr-x   1 mapr mapr          0 2014-11-05 11:36 /var/mapr/local/mapred/taskTracker/fidservers

The above line appears in all my other 8 nodes except 1 node. How should i work over that??

Output:
<pre>
hadoop fs -ls /var/mapr/local/mapred/taskTracker

Found 4 items

drwxr-xr-x   - mapr mapr          1 2014-11-05 10:39 /var/mapr/local/mapred/taskTracker/output

drwxr-xr-x   - mapr mapr          1 2014-11-05 10:39 /var/mapr/local/mapred/taskTracker/output.U

drwxr-xr-x   - mapr mapr          1 2014-11-05 10:39 /var/mapr/local/mapred/taskTracker/spill

drwxr-xr-x   - mapr mapr          1 2014-11-05 10:39 /var/mapr/local/mapred/taskTracker/spill.U
</pre>
====================================================================

Following Error found only in: hadoop-mapr-tasktracker.out
<pre>
2014-11-04 16:13:29,5410 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861410432768 removeFid failed for 2058.13024.15555886, error 2

2014-11-04 16:13:29,5423 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861410432768 Rmdirs failed for dir output/job_201411041452_0017/attempt_201411041452_0017_m_000000, Lookup rpc error No such file or directory(2)

2014-11-04 16:13:29,5423 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861410432768 removeFid failed for 2058.13022.15555882, error 2

2014-11-04 16:16:38,0115 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861381625600 Rmdirs failed for dir spill/job_201411041452_0020/attempt_201411041452_0020_m_000000_, Lookup rpc error No such file or directory(2)

2014-11-04 16:16:38,0115 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861381625600 removeFid failed for 2058.13046.15555930, error 2

2014-11-04 16:16:38,0130 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861381625600 Rmdirs failed for dir output/job_201411041452_0020/attempt_201411041452_0020_m_000000, Lookup rpc error No such file or directory(2)

2014-11-04 16:16:38,0131 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861381625600 removeFid failed for 2058.13044.15555926, error 2

2014-11-04 16:16:47,0639 ERROR Client fs/client/fileclient/cc/client.cc:1362 Thread: 139861404079872 Rmdir failed for dir spill/job_201411041452_0020/attempt_201411041452_0020_m_000000_, error Invalid argument(22)

2014-11-04 16:16:47,0639 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861404079872 removeFid failed for 2058.13046.15555930, error 22

2014-11-04 16:16:47,0658 ERROR Client fs/client/fileclient/cc/client.cc:1362 Thread: 139861404079872 Rmdir failed for dir output/job_201411041452_0020/attempt_201411041452_0020_m_000000, error Invalid argument(22)

2014-11-04 16:16:47,0658 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861404079872 removeFid failed for 2058.13044.15555926, error 22

2014-11-04 16:16:48,5701 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861400921856 Rmdirs failed for dir spill/job_201411041452_0020/attempt_201411041452_0020_m_000000_, Lookup rpc
error No such file or directory(2)

2014-11-04 16:16:48,5701 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861400921856 removeFid failed for 2058.13046.15555930, error 2

2014-11-04 16:16:48,5712 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861400921856 Rmdirs failed for dir output/job_201411041452_0020/attempt_201411041452_0020_m_000000, Lookup rpc error No such file or directory(2)

2014-11-04 16:16:48,5713 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861400921856 removeFid failed for 2058.13044.15555926, error 2

2014-11-04 16:17:56,2930 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861400921856 Rmdirs failed for dir spill/job_201411041452_0021/attempt_201411041452_0021_m_000000_, Lookup rpc error No such file or directory(2)

2014-11-04 16:17:56,2931 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861400921856 removeFid failed for 2058.12806.15555974, error 2

2014-11-04 16:17:56,2945 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861400921856 Rmdirs failed for dir output/job_201411041452_0021/attempt_201411041452_0021_m_000000, Lookup rpc error No such file or directory(2)

2014-11-04 16:17:56,2946 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861400921856 removeFid failed for 2058.12800.15555970, error 2

2014-11-04 16:18:05,2332 ERROR Client fs/client/fileclient/cc/client.cc:1362 Thread: 139861404079872 Rmdir failed for dir spill/job_201411041452_0021/attempt_201411041452_0021_m_000000_, error Invalid argument(22)

2014-11-04 16:18:05,2332 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861404079872 removeFid failed for 2058.12806.15555974, error 22

2014-11-04 16:18:05,2348 ERROR Client fs/client/fileclient/cc/client.cc:1362 Thread: 139861404079872 Rmdir failed for dir output/job_201411041452_0021/attempt_201411041452_0021_m_000000, error Invalid argument(22)

2014-11-04 16:18:05,2348 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861404079872 removeFid failed for 2058.12800.15555970, error 22

2014-11-04 16:18:07,3430 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861410432768 Rmdirs failed for dir spill/job_201411041452_0021/attempt_201411041452_0021_m_000000_, Lookup rpc error No such file or directory(2)

2014-11-04 16:18:07,3431 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861410432768 removeFid failed for 2058.12806.15555974, error 2

2014-11-04 16:18:07,3443 ERROR Client fs/client/fileclient/cc/client.cc:1394 Thread: 139861410432768 Rmdirs failed for dir output/job_201411041452_0021/attempt_201411041452_0021_m_000000, Lookup rpc error No such file or directory(2)

2014-11-04 16:18:07,3444 ERROR JniCommon fs/client/fileclient/cc/jni_common.cc:4483 Thread: 139861410432768 removeFid failed for 2058.12800.15555970, error 2
</pre>

Outcomes