AnsweredAssumed Answered

Running Hadoop Commands using MapR Client on Mac

Question asked by Rachel Silver Employee on Apr 26, 2017
Latest reply on May 9, 2017 by aalvarez

Hey, MapR support: after following the directions here, I'm unable to run simple Hadoop commands like "hadoop fs -ls /" from my Mac using the MapR client for Hadoop. 

 

The errors are: 

  • ERROR Cidcache fs/client/fileclient/cc/cidcache.cc:2192 Thread: 123145304514560 MoveToNextCldb: No CLDB entries, cannot run, sleeping 5 seconds!
  • ERROR Client fs/client/fileclient/cc/client.cc:1205 Thread: 123145304514560 Failed to initialize client for cluster rsilver.cluster.com, error Operation timed out(60)
    ls: Could not create FileClient

 

This is for MapR 5.2.1+MEP 3.0

 

The steps that I have followed: 

 

  • Downloaded the MapR client for Mac: mapr-client-5.2.1.42646GA-1.x86_64.tar.gz
  • Set Environment Variables in ~/.bash_profile:
    • export JAVA_HOME="$(/usr/libexec/java_home)"
      export MAPR_HOME=/opt/mapr
      HADOOP_HOME=$MAPR_HOME/hadoop/hadoop-2.7.0
      HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
      YARN_CONF_DIR=$HADOOP_CONF_DIR
      HADOOP_USER_NAME=mapr
  • Added symlinks:
    • ln -s /opt/mapr/hadoop/hadoop-0.20.2/bin/hadoop /usr/local/bin/hadoop1
    • ln -s /opt/mapr/hadoop/hadoop-2.7.0/bin/hadoop /usr/local/bin/hadoop2
    • ln -s /opt/mapr/bin/hadoop /usr/local/bin/hadoop
  • Set path in ~/.bash_profile
    • export PATH=/opt/mapr/bin:/opt/mapr/hadoop/hadoop-2.7.0/bin:/opt/mapr/hadoop/hadoop-0.20.2/bin:${PATH}
  • Added the following to local /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/mapred-site.xml
    • <property>
      <name>mapreduce.app-submission.cross-platform</name>
      <value>true</value>
      </property>
  • Added the following to local /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml
    • <property>
      <name>hadoop.spoofed.user.uid</name>
      <value>5000</value>
      </property>
      <property>
      <name>hadoop.spoofed.user.gid</name>
      <value>5000</value>
      </property>
      <property>
      <name>hadoop.spoofed.user.username</name>
      <value>mapr</value>
      </property>

 

What can I do to troubleshoot these errors?

Outcomes