AnsweredAssumed Answered

cannot run pig in local mode

Question asked by lei_tang on Jan 21, 2013
Latest reply on Feb 8, 2013 by lei_tang
Hi we are using the mapr hadoop distribution. I just installed mapr-pig-0.9.0.
Pig works fine in mapreduce mode, but does not work in local mode.

when I tried to run pig in local mode, I got the following error. 
I believe this is some issue with the configuration. Could you please help? 

I double checked. the file 'toy.txt' is there. 

    *[ltang@dev-trgt00 tmp]$ /opt/mapr/bin/pig -x local*
    
    2013-01-21 16:17:48,821 [main] INFO  org.apache.pig.Main - Logging error messages to: /data1/home/ltang01/tmp/pig_1358813868819.log
    2013-01-21 16:17:49,158 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
    
    grunt> A = load 'toy.txt';
    
    grunt> dump A;
    
    2013-01-21 16:18:25,559 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
    2013-01-21 16:18:25,725 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
    2013-01-21 16:18:25,761 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
    2013-01-21 16:18:25,762 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
    2013-01-21 16:18:25,788 [main] INFO  org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
    2013-01-21 16:18:25,813 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
    2013-01-21 16:18:25,848 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
    2013-01-21 16:18:30,876 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
    2013-01-21 16:18:30,915 [main] INFO  org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    2013-01-21 16:18:30,922 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
    2013-01-21 16:18:30,941 [Thread-3] INFO  org.apache.hadoop.util.NativeCodeLoader - Loaded the native-hadoop library
    2013-01-21 16:18:31,423 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
    2013-01-21 16:18:31,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job null has failed! Stop running all dependent jobs
    2013-01-21 16:18:31,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
    2013-01-21 16:18:31,441 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backend error: ENOENT: No such file or directory
    2013-01-21 16:18:31,441 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
    2013-01-21 16:18:31,441 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
    2013-01-21 16:18:31,443 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
    
    HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
    0.20.2-dev      0.9.1-SNAPSHOT  ltang01 2013-01-21 16:18:25     2013-01-21 16:18:31     UNKNOWN
    
    Failed!
    
    Failed Jobs:
    JobId   Alias   Feature Message Outputs
    N/A     A       MAP_ONLY        Message: ENOENT: No such file or directory
            at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
            at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513)
            at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325)
            at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325)
            at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
            at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
            at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873)
            at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:396)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109)
            at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867)
            at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841)
            at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
            at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
            at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
            at java.lang.Thread.run(Thread.java:662)
            file:/tmp/temp-859096669/tmp-120753216,
    
    Input(s):
    Failed to read data from "file:///data1/home/ltang01/tmp/toy.txt"
    
    Output(s):
    Failed to produce result in "file:/tmp/temp-859096669/tmp-120753216"
    
    Job DAG:
    null


    2013-01-21 16:18:31,443 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
    2013-01-21 16:18:31,449 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias A
    Details at logfile: /data1/home/ltang01/tmp/pig_1358813868819.log
    

Outcomes