AnsweredAssumed Answered

Latest HBase / Yarn versions are in conflict ???

Question asked by tc_dev on Aug 28, 2015
Latest reply on Aug 31, 2015 by tc_dev
I just went ahead with upgrade to the latest release of everything, including v5.0.0 core, v0.98.12 HBase and v2.7.0 MapReduce/ YARN

At first, the upgrade went seemingly fine. All packages installed (upgraded) fine without a problem, I manually edited ```/opt/mapr/conf/hadoop_version``` file to set YARN version and relaunched the cluster with calls to ```configure.sh``` (setting CLDB and Zookeeper nodes explicitly, rather than with -R option). The cluster went up fine and the Web UI reported ```Yarn MapReduce (2.7.0)``` in the upper right corner.

However, after running the cluster for a little, I noticed that the indicator switched back to ```Yarn MapReduce (2.5.1)```. When I looked back at ```/opt/mapr/conf/hadoop_version``` I saw that the file was overwritten with an older YARN version. I tried resetting it back to 2.7.0, rerunning ```configure.sh``` and observing the same behavior with 2.7.0 initially working and then soon reverting back to 2.5.1

To look for the source of the problem, I grepped for files which could harbor this old version number to be restored from ```grep -RF --exclude="*.log*" "2.5.1" /opt/mapr``` and it turned up among HBase patches: ```/opt/mapr/hbase/hbase-0.98.12/mapr-patches/0002-Configuring-HBase-build-for-0.98.12-mapr-1506-releas.patch:+    2.5.1-mapr-1501```

Questions:

 1. Does use of HBase 0.98.12 require use of YARN 2.5.1 or 2.7.0 is still an option?
 2. If v2.5.1 is required do I have to downgrade it manually, or whatever HBase does to reset it is sufficient? In my install YARN 2.5.1 only exists under ```/opt/mapr/hadoop/OLD_HADOOP_VERSIONS/``` since technically v2.7.0 was installed via yum . Not sure if it is expected to work there
 3. Could we have the docs updated to reflect HBase/YARN version dependencies? At the moment this subject is not covered anywhere as no other interdependencies between ecosystem components
 4. It would be helpful if such version conflicts are handled by install scipts (including yum upgrade)


Outcomes