AnsweredAssumed Answered

jobtracker dies soon after startup

Question asked by jeffcb on Jul 29, 2011
Latest reply on Aug 5, 2011 by srivas
After running configure.sh, starting zookeeper, and starting warden, the web interface shows that all the services are running (except nfs, see below).  But after a couple minutes, the jobtracker dies with this log message:

    2011-07-29 11:41:57,602 INFO org.apache.hadoop.mapred.JobTracker: Checking for jobtracker volume. If volume not present command will create and mount it. Command invoked is : /opt/mapr//server/createJTVolume.sh nimbus1.eng.vmware.com /var/mapr/cluster/mapred/jobTracker/system /var/mapr/cluster/mapred/jobTracker/system 
    2011-07-29 11:42:02,863 INFO org.apache.hadoop.mapred.JobTracker: Sucessfully created JobTracker volume and mounted at /var/mapr/cluster/mapred/jobTracker/system 
    2011-07-29 11:42:02,866 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 
    2011-07-29 11:42:02,867 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 
    2011-07-29 11:42:02,867 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 
    2011-07-29 11:42:02,867 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) 
    2011-07-29 11:42:02,868 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list 
    2011-07-29 11:42:02,909 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with owner as root 
    2011-07-29 11:42:02,928 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=JobTracker, port=9001 
    2011-07-29 11:42:02,931 INFO org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC Metrics with hostName=JobTracker, port=9001 
    2011-07-29 11:42:02,964 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 
    2011-07-29 11:42:03,002 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 
    2011-07-29 11:42:03,012 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50030 
    2011-07-29 11:42:03,014 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Address already in use 
        at sun.nio.ch.Net.bind(Native Method) 
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126) 
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) 
        at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) 
        at org.apache.hadoop.http.HttpServer.start(HttpServer.java:557) 
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2350) 
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2173) 
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:309) 
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:301) 
        at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:5346) 


Any ideas how to fix this?

Also, I have never been able to start the nfs server, so I removed that package.  Is it supported in M3?  Is it needed for the jobtracker or to run simple hadoop jobs?

Outcomes