AnsweredAssumed Answered

Exception while submitting Spark Job

Question asked by Velumani on Apr 24, 2016
Latest reply on Aug 30, 2016 by cmcdonald

Hi,

     I am getting this exception while running a simple Spark job. I saw a similar kind of issue posted in Stack Overflow ( http://stackoverflow.com/questions/33573587/apache-spark-wrong-akka-remote-netty-version/33636070 ). I excluded  netty dependencies in my POM.xml, still getting same exception.

Sample code  - GitHub - caroljmcdonald/SparkStreamingHBaseExample: Spark Streaming HBase Example

Spark Version - 1.5.2

Scala Version - 2.10.4

 

2016-04-24 23:25:04,421 ERROR [sparkDriver-akka.actor.default-dispatcher-4] actor.ActorSystemImpl: Uncaught fatal error from thread [sparkDriver-akka.remote.default-remote-dispatcher-7] shutting down ActorSystem [sparkDriver]

java.lang.VerifyError: (class: org/jboss/netty/channel/socket/nio/NioWorkerPool, method: createWorker signature: (Ljava/util/concurrent/Executor;)Lorg/jboss/netty/channel/socket/nio/AbstractNioWorker;) Wrong return type in function

        at akka.remote.transport.netty.NettyTransport.<init>(NettyTransport.scala:283)

        at akka.remote.transport.netty.NettyTransport.<init>(NettyTransport.scala:240)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78)

        at scala.util.Try$.apply(Try.scala:161)

        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73)

        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)

        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)

        at scala.util.Success.flatMap(Try.scala:200)

        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)

        at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:711)

        at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:703)

        at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:722)

        at scala.collection.Iterator$class.foreach(Iterator.scala:727)

        at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)

        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)

        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)

        at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:721)

        at akka.remote.EndpointManager.akka$remote$EndpointManager$$listens(Remoting.scala:703)

        at akka.remote.EndpointManager$$anonfun$receive$2.applyOrElse(Remoting.scala:491)

        at akka.actor.Actor$class.aroundReceive(Actor.scala:467)

        at akka.remote.EndpointManager.aroundReceive(Remoting.scala:394)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)

        at akka.actor.ActorCell.invoke(ActorCell.scala:487)

        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)

        at akka.dispatch.Mailbox.run(Mailbox.scala:220)

        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)

        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Outcomes