AnsweredAssumed Answered

mapwithstae word states lock for a particular key

Question asked by mahdi62b on Dec 7, 2016
Latest reply on Jan 6, 2017 by cmcdonald

I am just following the mapwithstae word count on databrick website..I have a conceptual question.

def trackStateFunc(batchTime: Time, key: String, value: Option[Int], state: State[Long]): Option[(String, Long)] = { val sum = value.getOrElse(0).toLong + state.getOption.getOrElse(0L) val output = (key, sum) state.update(sum) Some(output) }

I assume that when the streaming application enters this function, it locks the sates for a particular if an identical key is retrieved from other partitions and wanna update the doesn't read an old state value?