AnsweredAssumed Answered

java.io.InvalidClassException: scala.reflect.ClassTag$$anon$1; local class incompatible: stream classdesc serialVersionUID = -4937928798201944954, local class serialVersionUID = -8102093212602380348 : Spark

Question asked by madhureddy915 on Aug 6, 2015
Latest reply on Aug 7, 2015 by Ted Dunning
Hi Team ,

I'm getting below exception while counting the file rows :
Actually code is working fine if i executes shell in local mode :
But If we try shell with Yarn-client mode -- the below exceptions stack will be returning.
I've been investigated a little bit for the same problem. some are suggesting it's different JVM compatibility issue . It might be true because in other environment am using JDK6 it is working perfect in both local and Yarn-Client modes . In Another Environment am using JDK7 . here am facing the problem while running in yarn-client mode :


***java.io.InvalidClassException: scala.reflect.ClassTag$$anon$1; local class incompatible: stream classdesc serialVersionUID = -4937928798201944954, local class serialVersionUID = -8102093212602380348***

FYI  JAVA - :In Environment am facing the problem 

**java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470_27sr2fp10-20141218_02(SR2 FP10))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20141215_227395 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR2_20141215_1631_B227395
JIT  - tr.r13.java_20141003_74587.07
GC   - R27_Java727_SR2_20141215_1631_B227395_CMPRSS
J9CL - 20141215_227395)
JCL - 20141217_01 based on Oracle jdk7u75-b12**

***In another old Environment : it is working fine
java -version
java version "1.6.0_91"
Java(TM) SE Runtime Environment (build 1.6.0_91-b31)
Java HotSpot(TM) 64-Bit Server VM (build 20.91-b07, mixed mode***

From here I had moved the code to new Environment : If we suspects that it is incompatible version Issue then it shouldn't work in local mode too. Please clarify my questions:

Outcomes