AnsweredAssumed Answered

pig streaming does not work after upgrade to hadoop 1.0.3

Question asked by leitang on Jun 21, 2013
Latest reply on Jun 23, 2013 by leitang
Hi,
Our system just got upgraded to hadoop 1.0.3 (mapr client 2.1.3). After the upgrade, pig script with streaming does not work any more.  Here is a sample script. Tried both pig 0.9 and 0.10. Neither of them worked.


A = load 'toy.txt';
define nothing `cat`;
B = stream A through nothing;
store B into 'test';



Error message:
org.apache.pig.backend.executionengine.ExecException: ERROR 2083: Error while trying to get next result in POStream. at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNext(POStream.java:239) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:271) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:266) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:681) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2083: Error while trying to get next result in POStream. at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:308) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNext(POStream.java:222) ... 11 more Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2084: Error while running streaming binary. at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:285) ... 12 more Caused by: java.io.FileNotFoundException: File cat does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:395) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:257) at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:738) at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:713) at org.apache.pig.backend.hadoop.streaming.HadoopExecutableManager.configure(HadoopExecutableManager.java:84) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:280) ... 12 more

Outcomes