AnsweredAssumed Answered

Two problems with hadoop commands after MapR 3.1 > 5.1 upgrade

Question asked by andyt22 on May 5, 2016
Latest reply on May 9, 2016 by andyt22

I have just upgraded a 15 node M3 cluster that was originally running MapR 3.1 to 5.1 using the offline upgrade with manual steps procedure. All 13 data nodes seem to be working fine and a command like 'hadoop fs -ls /' works as expected. But on the two control nodes, one called 'howitzer' which has CLDB, NFS, zookeeper and historyserver functions and the other called 'bazooka' which hosts the MCS webserver plus zookeeper, some hadoop command fail with different errors; one control node gives these errors:

 

root@howitzer:~# hadoop fs

Error: Could not find or load main class org.apache.hadoop.fs.FsShell

and

root@howitzer:/# hadoop version

Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

 

Where should I set the classpath for hadoop?

 

The other control node correctly returns the hadoop version but fails to find core-site.xml:

 

root@bazooka:/# hadoop version

Hadoop 1.0.3-mapr-5.1.0

...

but

root@bazooka:/# hadoop fs

Exception in thread "main" java.lang.RuntimeException: core-site.xml not found

 

even though core-site.xml files do exist:

root@bazooka:/opt/mapr/hadoop# find ./ -name core-site.xml

./hadoop-2.7.0/etc/hadoop/core-site.xml

./hadoop-2.7.0/share/hadoop/common/templates/core-site.xml

./hadoop-0.20.2/conf/core-site.xml

./hadoop-0.20.2/conf.new/core-site.xml

 

Since the default Hadoop version I'm using now is 2.7.0, I assume the problem is with /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml not being on some path used by the hadoop command? Can anyone tell me where this should be set?

 

Thanks in advance for any pointers or suggestions,

Andy

Outcomes