Can someone please tell me how to submit a Pig script with job labels. I've tried the following with no luck:
SET mapreduce.job.label label_name; in Pig script
Passing -Dmapreduce.job.label=label_name from command line
Hi Sri Harsha Pemmaraju,
Where are you submitting pig job to? Check out https://cwiki.apache.org/confluence/display/OOZIE/Pig+Cookbook
if you are submitting the job through Oozie.
I'm trying to submit pig job from client node command line to Job Tracker as well as Resource manager.
This is how I'm running the job to MRv1:
pig -f scipt.pig -Dmapreduce.job.label=label_name -p jobname=testjob -p input=input_dir -p output=output_dir
We currently have processes running on MRv1 cluster which we plan to move to YARN. I was able to submit MapReduce applications using -Dmapreduce.job.label but I'd like to know the how pig jobs are submitted to MRv1 Job tracker as well as MRv2 Resource Manager.
Also, I've looked at the label based scheduling documentation. There are examples to submit MapReduce jobs using labels but not Pig jobs:
Label-based Scheduling for MRv1 Jobs
Hi Mathieu Dumoulin,
Could you please guide Sri and share your Pig knowledge?
Thank you very much!
Any update on this request?
Could you help? I think Mathieu is swamped at Strata.
Hello Sri Harsha Pemmaraju,
I've specified "mapreduce.job.label" property in pig script and submitted my test query:"grunt> SET mapreduce.job.label community;"
The "mapreduce.job.label" property was applied in both (MR1 and YARN) modes.
For checking it on MR1, I've used Job Tracker Web UI:
Click on"Jobid" in "Retired Jobs" table, in opened web page click on "JobConf" and check value for property "mapreduce.job.label".
For checking this value in YARN, I've used JobHistory Server UI:
Click on"Jobid" in "Retired Jobs" table, in opened web page click on "Configuration" and check value for property "mapreduce.job.label".
Thanks Murshid. I remember trying the same and it did not work for me. I will give it another shot today/tomorrow and will let you know if it is working as expected.
Does Murshid's suggestion work? Would greatly appreciate if you can provide the latest update.
Retrieving data ...