Is there anyway to pass YARN or MapReduce parameters (such as container mem allocation) to YARN jobs started by MapR admin utilities such as maprcli or utilties used on MapR-DB tables (CopyTable , etc) ?
for copytable i was able pass the memory parameters like mapreduce.map.memory.mb
mapr copytable -Dmapreduce.map.memory.mb=4096 -src /nishapps/user_profiles -dst /apps/user_profiles
Can you hint other mapr utilities here?
Inviting Nishchai Manjula to this conversation.
In the meantime, I wanted to share some related resources that might be of some help.
Regarding parameters for Yarn and mapreduce jobs:
Resource Allocation for Jobs and Applications
Regarding MapRCLI in MapR-DB:
MapR DB CLI Tutorial
Thank you for your reply.
Besides maprcli I can think of the hbase command used for HBase and MapR-DB Binary tables. So I guess that you can pass parameters like this with using -D definitions.
Thank you nishchai manjula , I tried your suggestion which makes sense on how to pass properties for the underlying MapReduce Job. However, I am using MapR-DB Binary tables and mapr documentation suggests to run it with the hbase CLI. I pass parameters using -D definitions (i.e -D=mapreduce.map.memory.mb=4096) however I cannot find the updated values in the configuration page when the MR job is running.
What i am currently running is this :
hbase -Dmapreduce.map.memory.mb=2048 -Dmapreduce.task.timeout=1200000 com.mapr.fs.hbase.tools.mapreduce.CopyTable -src /src_tbl -dst /dst_tbl -bulkload true
So far i want to deal with the timeouts so I doubled the expected running time.
Important : place -D definitions after the class name
Retrieving data ...