AnsweredAssumed Answered

Spark job stuck at collect in DEV362 Lab 10.3

Question asked by CindyXL on Apr 12, 2017
Latest reply on Apr 24, 2017 by CindyXL

Hi there,

 

I'm doing DEV362 Lession 10 Lab 10.3 at the moment.

When I tried the step below on page L10-15 of the lab guide:

flightsRDD.map(flight => flight.origin).distinct.collect.foreach(x
=> { originMap += (x -> index1); index1 += 1 })

 

The spark job stage 1 seems to be stuck. I killed the stage from Spark job console at 20mins and tried again. It's now been running for 10mins again. Is this normal? I'm running the sandbox on virtualbox for windows 7.

 

How do I troubleshoot?

 

Any tips would be greatly appreciated.

 

Thank you

Cindy

Outcomes