AnsweredAssumed Answered

Single node install failures (TaskTracker, NFS, HBase)

Question asked by tc_dev on Mar 4, 2012
Latest reply on Mar 5, 2012 by tc_dev
After going through single node install steps from the documentation there are a few problems left.

Specifically the admin UI gives alerts that TaskTracker and NFS services fail to start and HBase connection fails too.

Here is the error readout from the relevant logs -

============================================= 
/opt/mapr/hadoop/hadoop-0.20.2/logs/hadoop-root-tasktracker-XXXXXX.log
============================================= 

sun.cpu.endian: little
hadoop.policy.file: hadoop-policy.xml
sun.cpu.isalist:
------------------------------------------------------------*/
2012-03-04 21:51:33,763 INFO org.apache.hadoop.mapred.TaskTracker: /tmp is tmpfs. Java Hotspot Instrumentation will be enabled by default 
2012-03-04 21:51:33,764 INFO org.apache.hadoop.mapred.TaskTracker: Cleaning up config files from the job history folder 
2012-03-04 21:51:33,766 INFO org.apache.hadoop.mapred.TaskTracker: TT local config  is /opt/mapr/hadoop/hadoop-0.20.2/conf/mapred-site.xml 
2012-03-04 21:51:33,766 INFO org.apache.hadoop.mapred.TaskTracker: Loading resource properties file : /opt/mapr//logs/cpu_mem_disk 
2012-03-04 21:51:33,766 INFO org.apache.hadoop.mapred.TaskTracker: Physical memory reserved for mapreduce tasks = 915406848 bytes 
2012-03-04 21:51:33,766 INFO org.apache.hadoop.mapred.TaskTracker: CPUS: 8 
2012-03-04 21:51:33,767 INFO org.apache.hadoop.mapred.TaskTracker: Total MEM: 3.8588066GB 
2012-03-04 21:51:33,767 INFO org.apache.hadoop.mapred.TaskTracker: Reserved MEM: 673MB 
2012-03-04 21:51:33,767 INFO org.apache.hadoop.mapred.TaskTracker: Reserved MEM for Ephemeral slots 200 
2012-03-04 21:51:33,767 INFO org.apache.hadoop.mapred.TaskTracker: DISKS: 1 
2012-03-04 21:51:33,784 INFO org.apache.hadoop.mapred.TaskTracker: Map slots 6, Default heapsize for map task 60 mb 
2012-03-04 21:51:33,784 INFO org.apache.hadoop.mapred.TaskTracker: Reduce slots 4, Default heapsize for reduce task 78 mb 
2012-03-04 21:51:33,784 INFO org.apache.hadoop.mapred.TaskTracker: Ephemeral slots 1, memory given for each ephemeral slot 200 mb 
2012-03-04 21:51:33,784 INFO org.apache.hadoop.mapred.TaskTracker: Prefetch map slots 3 
2012-03-04 21:51:33,855 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 
2012-03-04 21:51:33,926 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 
2012-03-04 21:51:33,937 WARN org.apache.hadoop.mapred.LinuxTaskController: Exit code from checking binary permissions is : 22 
2012-03-04 21:51:33,937 INFO org.apache.hadoop.mapred.TaskController: Reading task controller config from /opt/mapr/hadoop/hadoop-0.20.2/conf/taskcontroller.cfg
2012-03-04 21:51:33,937 INFO org.apache.hadoop.mapred.TaskController: The task-controller binary should not have write or execute for others. 
2012-03-04 21:51:33,938 INFO org.apache.hadoop.mapred.TaskController: Invalid permissions on task-controller binary. 
2012-03-04 21:51:33,938 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start TaskTracker because java.io.IOException: Task controller setup failed because of invalidpermissions/ownership with exit code 22 
at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:148) 
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:2149) 
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:5216) 
Caused by: org.apache.hadoop.util.Shell$ExitCodeException:  
at org.apache.hadoop.util.Shell.runCommand(Shell.java:322) 
at org.apache.hadoop.util.Shell.run(Shell.java:249) 
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:442) 
at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:142)  
... 2 more 

2012-03-04 21:51:33,940 INFO org.apache.hadoop.mapred.TaskTracker: SHUTDOWN_MSG:  
/************************************************************ 
SHUTDOWN_MSG: Shutting down TaskTracker at XXXXXX/127.0.0.1 
************************************************************/ 

============================================= 
/opt/mapr/logs/nfsserver.log
============================================= 

2012-03-05 01:44:26,1514 INFO nfsserver[13470] fs/nfsd/mount.cc:1968 Export info: /mapr (rw) 
2012-03-05 01:44:26,1517 INFO nfsserver[13470] fs/nfsd/mount.cc:1693 CLDB info: 127.0.0.1:7222  
2012-03-05 01:44:26,1523 INFO nfsserver[13470] fs/nfsd/nfsha.cc:399 hostname: XXXXXX, hostid: 0x34dff6b8b8ed6ea1 
2012-03-05 01:44:26,1525 INFO nfsserver[13470] fs/nfsd/nfsha.cc:153 No carrier-sense on eth1, skipped 
2012-03-05 01:44:26,1525 INFO nfsserver[13470] fs/nfsd/nfsha.cc:153 No carrier-sense on sit0, skipped 
2012-03-05 01:44:26,1527 INFO nfsserver[13470] fs/nfsd/requesthandle.cc:410 found NFS_HEAPSIZE env var: 119 
2012-03-05 01:44:26,2357 INFO nfsserver[13470] fs/nfsd/main.cc:487 NFS server started ... pid=13470, uid=0 
2012-03-05 01:44:26,2371 INFO nfsserver[13470] fs/nfsd/nfsha.cc:887 exiting: No license to run NFS server 

============================================= 
Additionally HBase shell example from the documentation fails as well -

HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.90.4, rUnknown, Wed Nov  9 17:35:00 PST 2011

hbase(main):001:0> create 'weblog', 'stats'

ERROR: org.apache.hadoop.hbase.MasterNotRunningException: null

Here is some help for this command:
Create table; pass table name, a dictionary of specifications per
column family, and optionally a dictionary of table configuration.
Dictionaries are described below in the GENERAL NOTES section.
Examples:

  hbase> create 't1', {NAME => 'f1', VERSIONS => 5}
  hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'}
  hbase> # The above in shorthand would be the following:
  hbase> create 't1', 'f1', 'f2', 'f3'
  hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true}

============================================= 
  
What could be causing these problems?

Outcomes