AnsweredAssumed Answered

apache pig with mapr client distribution

Question asked by matt on Oct 4, 2011
Latest reply on Oct 4, 2011 by richa
I've installed the mapr client distro on mac os 10.6 and I can query the cluster with hadoop commands. Now I'm trying to query the cluster with Pig 0.9.0 on that same client node. I've set:
<code>
 PIG_CONF_DIR=/opt/mapr/hadoop/hadoop-0.20.2/conf
</code>

and it seems to read that correctly because I get the following output (and contents of log):
<code>
(37) 13:55:45 $ pig
2011-10-04 13:55:48,630 [main] INFO  org.apache.pig.Main - Logging error messages to: /Users/matt/Shed/pig_1317750948628.log
2011-10-04 13:55:48,855 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: maprfs:///
2011-10-04 13:55:48,908 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error. Failed to create DataStorage
Details at logfile: /Users/matt/Shed/pig_1317750948628.log
(38) 13:55:48 $ cat pig_1317750948628.log
Error before Pig is launched
ERROR 2999: Unexpected internal error. Failed to create DataStorage
java.lang.RuntimeException: Failed to create DataStorage
at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:196)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:116)
at org.apache.pig.impl.PigContext.connect(PigContext.java:184)
at org.apache.pig.PigServer.<init>(PigServer.java:243)
at org.apache.pig.PigServer.<init>(PigServer.java:228)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:46)
at org.apache.pig.Main.run(Main.java:484)
at org.apache.pig.Main.main(Main.java:108)
Caused by: java.io.IOException: No FileSystem for scheme: maprfs
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1375)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
... 9 more

</code>

What jar/libraries do I need to configure pig with to be able to use the client installation?

Outcomes