AnsweredAssumed Answered

Apache Spark YARN mode startup takes too long from the Edge Node :/

Question asked by ghalbi on Aug 8, 2017
Latest reply on Aug 22, 2017 by ghalbi

Hi,

 

I'm using Apache toree kernels( Scala, PysparK ) to submit a spark job on yarn  remotely.

-Edge Node contains ( Jupyter Notebook ) on Azure 4cores, 14Go ram.

-MapRSandBox 5.2 on Cloud Azure: 4 cores, 14Go ram.

 

it's take too long to create a sqlcontext from edge node than directly on MapRSandBox !!

Is it a question of performance ??

 

Thank you,

 

Mathieu Dumoulin

Outcomes