AnsweredAssumed Answered

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.conf.CoreDefaultProperties not found  

Question asked by bgajjela on Jul 2, 2016
Latest reply on Jul 5, 2016 by bgajjela

Today I was upgrading MapR 5.1 in sandbox which seems to be having below issues:

 

1) we have mapr-patch-4.1.0.31175.GA-34319.x86_64 , so once we did yum upgrade it was failing due to the dependency.

 

--> Finished Dependency Resolution

 

Error: Package: mapr-patch-4.1.0.31175.GA-34319.x86_64 (installed)

Requires: mapr-core-internal = 4.1.0.31175.GA-1

Removing: mapr-core-internal-4.1.0.31175.GA-1.x86_64 (@maprtech)

mapr-core-internal = 4.1.0.31175.GA-1

Updated By: mapr-core-internal-5.1.0.37549.GA-1.x86_64 (maprtech)

mapr-core-internal = 5.1.0.37549.GA-1

 

 

2) So what i did was tried with --skip-broken so mapr-core-internal is skipped and rest of the packages are installed.

 

I removed the patch

 

$ rpm -ea mapr-patch-4.1.0.31175.GA-34319.x86_64

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-api-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-api-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-common-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-common-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/tools/lib/hadoop-streaming-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/tools/lib/hadoop-streaming-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1-mapr-1503.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1-mapr-1503-tests.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1-mapr-1503-tests.jar': No such file or directory

mv: cannot move `/opt/mapr/.patch/hadoop/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1-mapr-1503.jar.O' to `/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1-mapr-1503.jar': No such file or directory

 

 

 

 

and reinstalled mapr-core-internal and below is the snippet from the install logs

 

 

Bringing down all MapR configured virtual IPs

  Cleanup    : mapr-core-internal-4.1.0.31175.GA-1.x86_64 2/2

/var/tmp/rpm-tmp.M1ilHy: line 34: [: =: unary operator expected

/var/tmp/rpm-tmp.M1ilHy: line 38: [: =: unary operator expected

  Verifying  : mapr-core-internal-5.1.0.37549.GA-1.x86_64 1/2

  Verifying  : mapr-core-internal-4.1.0.31175.GA-1.x86_64 2/2

 

Updated:

  mapr-core-internal.x86_64 0:5.1.0.37549.GA-1

 

 

3) once This part is completed in terms of upgrade/install , i started cluster and ran below commands.

 

$hadoop conf

Try 'hadoop conf-details' for more detailed configuration.

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory

        at org.apache.hadoop.security.authentication.util.KerberosName.<clinit>(KerberosName.java:43)

        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)

        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:197)

        at org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:387)

        at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:381)

        at org.apache.hadoop.mapreduce.conf.MapReduceDefaultProperties.<clinit>(MapReduceDefaultProperties.java:111)

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:264)

        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2147)

        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2362)

        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2579)

        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2531)

        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2444)

        at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:642)

        at org.apache.hadoop.conf.Configuration.asXmlDocument(Configuration.java:2773)

        at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2742)

        at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2732)

        at org.apache.hadoop.conf.Configuration.main(Configuration.java:2911)

        at org.apache.hadoop.mapreduce.util.ConfigUtil.main(ConfigUtil.java:550)

Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory

        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)

        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

        ... 19 more

 

$ hadoop mfs

Error: Could not find or load main class com.mapr.fs.clicommands.MapRCliComma

 

hadoop fs -ls /

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.conf.CoreDefaultProperties not found

        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2612)

        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2531)

        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2444)

        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1156)

        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1128)

        at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1464)

        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:321)

        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:487)

        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)

        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

        at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)

 

 

 

$ hadoop classpath

/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/*:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/*:/contrib/capacity-scheduler/*.jar:/opt/mapr/lib/kvstore*.jar:/opt/mapr/lib/libprotodefs*.jar:/opt/mapr/lib/baseutils*.jar:/opt/mapr/lib/maprutil*.jar:/opt/mapr/lib/json-20080701.jar:/opt/mapr/lib/flexjson-2.1.jar

 

 

It seems like the class-path is broken when the patch was removed. or Iam missing anything here. Please let me know what is the safe procedure to perform 5.1 upgrade with previous version of patch.

Outcomes