AnsweredAssumed Answered

cannot run pig in local mode

Question asked by lei_tang on Jan 21, 2013
Latest reply on Feb 8, 2013 by lei_tang
Hi we are using the mapr hadoop distribution. I just installed mapr-pig-0.9.0.
Pig works fine in mapreduce mode, but does not work in local mode.

when I tried to run pig in local mode, I got the following error. 
I believe this is some issue with the configuration. Could you please help? 

I double checked. the file 'toy.txt' is there. 

    *[ltang@dev-trgt00 tmp]$ /opt/mapr/bin/pig -x local*
    2013-01-21 16:17:48,821 [main] INFO  org.apache.pig.Main - Logging error messages to: /data1/home/ltang01/tmp/pig_1358813868819.log
    2013-01-21 16:17:49,158 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
    grunt> A = load 'toy.txt';
    grunt> dump A;
    2013-01-21 16:18:25,559 [main] INFO - Pig features used in the script: UNKNOWN
    2013-01-21 16:18:25,725 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
    2013-01-21 16:18:25,761 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
    2013-01-21 16:18:25,762 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
    2013-01-21 16:18:25,788 [main] INFO  org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
    2013-01-21 16:18:25,813 [main] INFO - Pig script settings are added to the job
    2013-01-21 16:18:25,848 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
    2013-01-21 16:18:30,876 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
    2013-01-21 16:18:30,915 [main] INFO  org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
    2013-01-21 16:18:30,922 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
    2013-01-21 16:18:30,941 [Thread-3] INFO  org.apache.hadoop.util.NativeCodeLoader - Loaded the native-hadoop library
    2013-01-21 16:18:31,423 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
    2013-01-21 16:18:31,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job null has failed! Stop running all dependent jobs
    2013-01-21 16:18:31,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
    2013-01-21 16:18:31,441 [main] ERROR - ERROR 2997: Unable to recreate exception from backend error: ENOENT: No such file or directory
    2013-01-21 16:18:31,441 [main] ERROR - 1 map reduce job(s) failed!
    2013-01-21 16:18:31,441 [main] INFO - Detected Local mode. Stats reported below may be incomplete
    2013-01-21 16:18:31,443 [main] INFO - Script Statistics:
    HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
    0.20.2-dev      0.9.1-SNAPSHOT  ltang01 2013-01-21 16:18:25     2013-01-21 16:18:31     UNKNOWN
    Failed Jobs:
    JobId   Alias   Feature Message Outputs
    N/A     A       MAP_ONLY        Message: ENOENT: No such file or directory
            at Method)
            at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(
            at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(
            at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(
            at org.apache.hadoop.fs.FilterFileSystem.mkdirs(
            at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(
            at org.apache.hadoop.mapred.JobClient$
            at org.apache.hadoop.mapred.JobClient$
            at Method)
            at org.apache.hadoop.mapred.JobClient.submitJobInternal(
            at org.apache.hadoop.mapred.JobClient.submitJob(
            at org.apache.hadoop.mapred.jobcontrol.Job.submit(
            at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(
    Failed to read data from "file:///data1/home/ltang01/tmp/toy.txt"
    Failed to produce result in "file:/tmp/temp-859096669/tmp-120753216"
    Job DAG:

    2013-01-21 16:18:31,443 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
    2013-01-21 16:18:31,449 [main] ERROR - ERROR 1066: Unable to open iterator for alias A
    Details at logfile: /data1/home/ltang01/tmp/pig_1358813868819.log