AnsweredAssumed Answered

Function inside foreach - Spark

Question asked by Preetha on Dec 23, 2016
Latest reply on Dec 28, 2016 by raju.bairishetti

I am going through the lab exercise 10.3 Analyze Flight Delays with Spark Machine Learning Scenario under chapter 10 for  Machine learning.

My question is from Page 147 Lab Guide for 

My question is what does the green highlighted section do in the below code snippet.

 

Code Snippet

//Below we transform non-numeric features into numeric values. For example, the carrier AA is the number 6.
// The originating airport ATL is 273.
// create airports RDD with ID and Name

 

var carrierMap: Map[String, Int] = Map()
var index: Int = 0
flightsRDD.map(flight => flight.carrier).distinct.collect.foreach(x => { carrierMap += (x -> index); index += 1 }) carrierMap.toString
//res2: String = Map(DL -> 5, F9 -> 10, US -> 9, OO

Outcomes