I'm trying to use Spark ML with Structured Streaming, and seems like it requires Spark 2.3.0 or above. Is there any estimate when MEP with Spark 2.3.0 is going to be released? Thanks!
IIRC, you should be able to use Spark Structured Streaming in earlier releases like spark 2.1
With respect to Spark 2.3 support you will want to use the MapR release and AFAIK 6.1 is coming out soon. (But I couldn't tell you when soon will be. Definitely not in geodesic time, but something faster. ;-) )
Hi Michael, how are you?
Yes, Structured Streaming is available already, but I specifically mentioned ML. After writing the initial message I dug a bit more. Apparently, using ML prediction functionality on structured streaming dataframes becomes available only in Spark 2.3.0 - it's specifically mentioned in 2.3.0 release notes.
I can only hope the next MEP is gonna have 2.3.0 inside, but until then I have to rewrite my app in old Spark streaming Guess it will also allow me to dump streaming data into MapR-DB, too (couldn't make it work in Structured).
Retrieving data ...