AnsweredAssumed Answered

mapwithstae word count..do states lock for a particular key

Question asked by mahdi62b on Dec 7, 2016
Latest reply on Jan 6, 2017 by cmcdonald

I am just following the mapwithstae word count on databrick website..I have a conceptual question.

def trackStateFunc(batchTime: Time, key: String, value: Option[Int], state: State[Long]): Option[(String, Long)] = { val sum = value.getOrElse(0).toLong + state.getOption.getOrElse(0L) val output = (key, sum) state.update(sum) Some(output) }

I assume that when the streaming application enters this function, it locks the sates for a particular key..so if an identical key is retrieved from other partitions and wanna update the sates..it doesn't read an old state value? 

Outcomes