AnsweredAssumed Answered

Issue with Livy and sparkR

Question asked by luca_guerra on Sep 21, 2016
Latest reply on Sep 21, 2016 by luca_guerra

Hello,

I'm working with: MapR = 5.2, Hue = 3.9.0, Spark = 1.6.1 (with YARN in cluster mode) on CentOS 6.7, I installed Livy from MapR installer, Scala notebook works correctly but when I create an R notebook I have this error message:

SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/filecache/176/livy-assembly.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 16/09/21 14:33:45 ERROR ScalatraListener: Failed to initialize scalatra application at  java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)      at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)      at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)      at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)      at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)      at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)      at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)      at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.server.Server.start(Server.java:387)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)      at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)      at com.cloudera.hue.livy.repl.Main.main(Main.scala)      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:606)      at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) Caused by: java.io.IOException: error=2, No such file or directory      at java.lang.UNIXProcess.forkAndExec(Native Method)      at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)      at java.lang.ProcessImpl.start(ProcessImpl.java:130)      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)      ... 28 more 16/09/21 14:33:45 WARN AbstractLifeCycle: FAILED c.c.h.j.s.ServletContextHandler@2f165f4a{/,file:/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1473064179684_0225/container_e04_1473064179684_0225_02_000001/src/main/com/cloudera/hue/livy/repl,STARTING}: java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)      at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)      at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)      at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)      at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)      at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)      at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)      at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.server.Server.start(Server.java:387)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)      at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)      at com.cloudera.hue.livy.repl.Main.main(Main.scala)      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:606)      at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) Caused by: java.io.IOException: error=2, No such file or directory      at java.lang.UNIXProcess.forkAndExec(Native Method)      at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)      at java.lang.ProcessImpl.start(ProcessImpl.java:130)      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)      ... 28 more 16/09/21 14:33:45 WARN AbstractLifeCycle: FAILED com.cloudera.hue.jetty.server.handler.HandlerCollection@65ca0b6c[c.c.h.j.s.ServletContextHandler@2f165f4a{/,file:/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1473064179684_0225/container_e04_1473064179684_0225_02_000001/src/main/com/cloudera/hue/livy/repl,STARTING}, com.cloudera.hue.jetty.server.handler.RequestLogHandler@33b413a9]: java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)      at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)      at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)      at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)      at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)      at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)      at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)      at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.server.Server.start(Server.java:387)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)      at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)      at com.cloudera.hue.livy.repl.Main.main(Main.scala)      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:606)      at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) Caused by: java.io.IOException: error=2, No such file or directory      at java.lang.UNIXProcess.forkAndExec(Native Method)      at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)      at java.lang.ProcessImpl.start(ProcessImpl.java:130)      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)      ... 28 more 16/09/21 14:33:45 WARN AbstractLifeCycle: FAILED com.cloudera.hue.jetty.server.Server@615c027c: java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)      at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)      at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)      at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)      at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)      at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)      at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)      at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.server.Server.start(Server.java:387)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)      at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)      at com.cloudera.hue.livy.repl.Main.main(Main.scala)      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:606)      at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) Caused by: java.io.IOException: error=2, No such file or directory      at java.lang.UNIXProcess.forkAndExec(Native Method)      at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)      at java.lang.ProcessImpl.start(ProcessImpl.java:130)      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)      ... 28 more 16/09/21 14:33:45 ERROR ApplicationMaster: User class threw exception: java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)      at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)      at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)      at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)      at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)      at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)      at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)      at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)      at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)      at com.cloudera.hue.jetty.server.Server.start(Server.java:387)      at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)      at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)      at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)      at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)      at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)      at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)      at com.cloudera.hue.livy.repl.Main.main(Main.scala)      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:606)      at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) Caused by: java.io.IOException: error=2, No such file or directory      at java.lang.UNIXProcess.forkAndExec(Native Method)      at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)      at java.lang.ProcessImpl.start(ProcessImpl.java:130)      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)      ... 28 more 16/09/21 14:33:55 ERROR ApplicationMaster: SparkContext did not initialize after waiting for 100000 ms. Please check earlier log output for errors. Failing the application.

I setted environment variable: 

PATH=$PATH:/opt/mapr/spark/spark-1.6.1/bin/
export SPARK_HOME=/opt/mapr/spark/spark-1.6.1/

for mapr and root users in file .bash_profile and installed R on all cluster nodes but the problem persist.

 

Thank you for the help,

Luca.

Outcomes