AnsweredAssumed Answered

Erratic behaviour of hadoop dfs -dus

Question asked by siddharth1988 on Apr 10, 2013
Latest reply on Apr 10, 2013 by Ted Dunning
We noticed that when we issue command like this:-
hadoop dfs -dus /tmp/* -- system return following

<pre>
dus: Cannot access /tmp/amit: No such file or directory.
dus: Cannot access /tmp/anatv1: No such file or directory.
dus: Cannot access /tmp/aniesh: No such file or directory.
dus: Cannot access /tmp/apundi: No such file or directory.
dus: Cannot access /tmp/config_ldap.tar: No such file or directory.
dus: Cannot access /tmp/hadoop-adabas: No such file or directory.
dus: Cannot access /tmp/hadoop-agoel1: No such file or directory.
</pre>

but if we issue the same command as follows :-
hadoop dfs -dus /mapr/silverluster/tmp/* -- or -- hadoop dfs -dus "/tmp/*"

it works fine and returns:-

<pre>
maprfs:/mapr/silvercluster/tmp/dfsouts  629370
maprfs:/mapr/silvercluster/tmp/ganan5   9056
maprfs:/mapr/silvercluster/tmp/gkassabo 0
maprfs:/mapr/silvercluster/tmp/hive-apundi      0
maprfs:/mapr/silvercluster/tmp/hive-bcheluva    0
maprfs:/mapr/silvercluster/tmp/hive-hadpsup     0
maprfs:/mapr/silvercluster/tmp/hive-root        0
maprfs:/mapr/silvercluster/tmp/hive-sshah62     0
</pre>

why is this behaviour in first place ? is there a bug in the API ?

Second question is, if I am in say my development cluster and I issue the same command as above for my production cluster my command works perfectly fine and lists me dus for a different cluster .

ie. from silvercluster if I issue following command
<code>
hadoop dfs -dus /mapr/platinumcluster/tmp/* 
</code>
 -- I get the results  ( Ideally it shouldnt allow me to access anything in platinum from silver but it does )

I beleive since the API queries the CLDB for this it requires the clustername and also the enlisting of that cluster in mapr-cluster.conf . But why does it accept values in quote ? This is a bug in API as when I issue the command then the API must understand that the command is for the maprfs not local file system.

Please elaborate !!!



PS:
Silvercluster -- Dev environment
Platinum -- Production Environment

Outcomes