AnsweredAssumed Answered

error running pig in local mode

Question asked by leitang on Jul 12, 2013
Latest reply on Jul 13, 2013 by gera
Hi,

It seems pig works fine in map reduce mode, but could not proceed in local mode. This error occurs after the mapr hadoop system upgrade to hadoop 1.0.3.  Tried both pig 0.9 and 0.10, and both mapr and apache version. The error stays consistently. It seems more related to configuration. 

Here is a toy pig script running in local mode:
<code>
grunt> [ltang01@dev-trgt00 tmp]$ pig -x local
2013-07-12 13:44:03,245 [main] INFO  org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
2013-07-12 13:44:03,246 [main] INFO  org.apache.pig.Main - Logging error messages to: /data1/home/ltang01/tmp/pig_1373661843241.log
2013-07-12 13:44:03,500 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
grunt> A = load 'toy.txt';
grunt> dump A;
</code>

<pre>
2013-07-12 13:44:21,381 [Thread-7] WARN  org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.ExceptionInInitializerError
        at org.apache.hadoop.mapred.CentralTaskLogUtil.<clinit>(CentralTaskLogUtil.java:62)
        at org.apache.hadoop.mapred.TaskStatus.<init>(TaskStatus.java:100)
        at org.apache.hadoop.mapred.MapTaskStatus.<init>(MapTaskStatus.java:29)
        at org.apache.hadoop.mapred.TaskStatus.createTaskStatus(TaskStatus.java:479)
        at org.apache.hadoop.mapred.Task.<init>(Task.java:194)
        at org.apache.hadoop.mapred.MapTask.<init>(MapTask.java:117)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193)
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.mapred.TaskTracker.getMapRHostname(TaskTracker.java:2770)
        at org.apache.hadoop.mapred.TaskTracker.initializeHostname(TaskTracker.java:1168)
        at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:220)
        ... 7 more
2013-07-12 13:44:21,594 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local_0001
2013-07-12 13:44:21,669 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.3.6--1, built on 09/07/2012 18:16 GMT
2013-07-12 13:44:21,670 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=dev-trgt00.sv.walmartlabs.com
2013-07-12 13:44:21,670 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.6.0_38
2013-07-12 13:44:21,670 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Sun Microsystems Inc.
2013-07-12 13:44:21,670 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/java/jdk1.6.0_38/jre
</pre>

this is the output from corresponding log file:
<pre>
[ltang01@dev-trgt00 tmp]$ less pig_1373661085227.log
Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias A

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias A
        at org.apache.pig.PigServer.openIterator(PigServer.java:857)
        at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.run(Main.java:490)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
        at org.apache.pig.PigServer.openIterator(PigServer.java:849)
        ... 12 more
================================================================================

</pre>
It looks like pig cannot identify the MapRhost in local mode.  Any idea?

Your comment is highly appreciated.
- Lei

Outcomes