AnsweredAssumed Answered

Unused V-Cores and Memory

Question asked by sourav_das on Sep 20, 2016
Latest reply on Oct 5, 2016 by MichaelSegel

Hi

I am executing SQOOP jobs in my Mapr Cluster.. I have a 6 node cluster with 2TB memory and 264 vcores. When I submit SQOOP jobs , I can see the jobs are taking time to execute ... Roughly , a 50-60Mill data from Oracle is taking 20 mins to load in maprfs ...When I see the application manager UI , the memory usage is just 10-15 GB and the vCore usage is also around 10-11 ... It seems to me that , I have more memory and vcores but the job is not consuming at that level ....

 

Is there a way , I can set these parameters for each sqoop job depending on the volume of data its trying to load ? 

Outcomes