AnsweredAssumed Answered

Preferred method for including third party libs for MR job

Question asked by dave_kincaid on May 21, 2013
Latest reply on May 22, 2013 by gera
What is the currently preferred method for including third party libraries for a MapReduce job. A search on the web seems to reveal three different options, but it's not clear what still actually works and what doesn't. So what is currently the preferred method of these three:

 1. Include the third party jars in a lib/ directory in the MR job jar file that is launched with "hadoop jar"
 2. Use -libjars on the command line with all the third party jars needed listed (which could be a lot)
 3. Unjar the third party libs and include the unjar'd files directly in the MR job jar file

Are there any others? I'm trying #1 right now and it's not working when using the MapR client to submit to a remote cluster. Does that method no longer work?

Thanks.

Outcomes