AnsweredAssumed Answered

Zeppelin docker storage defaults to git and web service doesnt start

Question asked by waynetheron on Feb 13, 2018
Latest reply on Feb 17, 2018 by Rachel Silver

following the instructions on Running the Zeppelin Docker Image 

 

docker command:

 

docker run -it -p 9995:9995 \
-e HOST_IP=10.112.36.176 -p 10000-10010:10000-10010 \
-e MAPR_CLUSTER=my.cluster.com -e MAPR_CLDB_HOSTS=10.112.35.191 \
-e MAPR_CONTAINER_USER=mapr -e MAPR_CONTAINER_PASSWORD=password -e MAPR_CONTAINER_GROUP=mapr -e MAPR_CONTAINER_UID=5000 -e MAPR_CONTAINER_GID=5000 -e MAPR_TICKETFILE_LOCATION=/tmp/maprticket_5000 -e MAPR_MOUNT_PATH=/mapr \
--cap-add  SYS_ADMIN --cap-add SYS_RESOURCE --device /dev/fuse \
-e MAPR_HS_HOST=10.112.35.191 -e ZEPPELIN_NOTEBOOK_DIR=/mapr/my.cluster.com/user/mapr/notebook \
-e MAPR_TZ=US/Pacific maprtech/data-science-refinery:v1.1_6.0.0_4.1.0_ubuntu16

 

everything seems to work but the website returns

 

HTTP ERROR: 503

Problem accessing /. Reason:

    Service Unavailable

Looking at the config "zeppelin-site.xml" it is pointing to git and as a result

<property>
  <name>zeppelin.notebook.storage</name>
  <value>org.apache.zeppelin.notebook.repo.GitNotebookRepo</value>
  <description>versioned notebook persistence layer implementation</description>
</property>

the following errors are logged in zeppelin where it can't create the notebook folder
defined in the docker run command ZEPPELIN_NOTEBOOK_DIR=/mapr/my.cluster.com/user/mapr/notebook.
This seems to crash the web app and here is the full error dump
 WARN [2018-02-13 13:09:57,405] ({main} NotebookRepoSync.java[<init>]:87) - Failed to initialize org.apache.zeppelin.notebook.repo.GitNotebookRepo notebook storage class
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.zeppelin.notebook.repo.NotebookRepoSync.<init>(NotebookRepoSync.java:83)
        at org.apache.zeppelin.server.ZeppelinServer.<init>(ZeppelinServer.java:148)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createSingletonInstance(CXFNonSpringJaxrsServlet.java:382)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createApplicationInstance(CXFNonSpringJaxrsServlet.java:454)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createServerFromApplication(CXFNonSpringJaxrsServlet.java:432)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:93)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
Caused by: org.apache.commons.vfs2.FileSystemException: Could not create folder "file:///mapr/my.cluster.com".
        at org.apache.commons.vfs2.provider.AbstractFileObject.createFolder(AbstractFileObject.java:999)
        at org.apache.commons.vfs2.provider.AbstractFileObject.createFolder(AbstractFileObject.java:982)
        at org.apache.commons.vfs2.provider.AbstractFileObject.createFolder(AbstractFileObject.java:982)
        at org.apache.commons.vfs2.provider.AbstractFileObject.createFolder(AbstractFileObject.java:982)
        at org.apache.zeppelin.notebook.repo.VFSNotebookRepo.setNotebookDirectory(VFSNotebookRepo.java:92)
        at org.apache.zeppelin.notebook.repo.VFSNotebookRepo.<init>(VFSNotebookRepo.java:69)
        at org.apache.zeppelin.notebook.repo.GitNotebookRepo.<init>(GitNotebookRepo.java:61)
        ... 35 more
Caused by: org.apache.commons.vfs2.FileSystemException: Could not create directory "/mapr/my.cluster.com".
        at org.apache.commons.vfs2.provider.local.LocalFile.doCreateFolder(LocalFile.java:153)
        at org.apache.commons.vfs2.provider.AbstractFileObject.createFolder(AbstractFileObject.java:988)
        ... 41 more
 INFO [2018-02-13 13:09:57,407] ({main} NotebookRepoSync.java[<init>]:92) - No storage could be initialized, using default org.apache.zeppelin.notebook.repo.GitNotebookRepo storage
 INFO [2018-02-13 13:09:57,408] ({main} VFSNotebookRepo.java[setNotebookDirectory]:91) - Notebook dir doesn't exist, create on is file:///mapr/my.cluster.com/user/mapr/notebook.
 WARN [2018-02-13 13:09:57,408] ({main} NotebookRepoSync.java[initializeDefaultStorage]:116) - Failed to initialize org.apache.zeppelin.notebook.repo.GitNotebookRepo notebook storage class java.lang.reflect.InvocationTargetException
 INFO [2018-02-13 13:09:57,548] ({main} NotebookAuthorization.java[loadFromFile]:96) - /opt/mapr/zeppelin/zeppelin-0.7.2/conf/notebook-authorization.json
 INFO [2018-02-13 13:09:57,549] ({main} Credentials.java[loadFromFile]:102) - /opt/mapr/zeppelin/zeppelin-0.7.2/conf/credentials.json
 INFO [2018-02-13 13:09:57,584] ({main} StdSchedulerFactory.java[instantiate]:1184) - Using default implementation for ThreadExecutor
 INFO [2018-02-13 13:09:57,588] ({main} SimpleThreadPool.java[initialize]:268) - Job execution threads will use class loader of thread: main
 INFO [2018-02-13 13:09:57,599] ({main} SchedulerSignalerImpl.java[<init>]:61) - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
 INFO [2018-02-13 13:09:57,600] ({main} QuartzScheduler.java[<init>]:240) - Quartz Scheduler v.2.2.1 created.
 INFO [2018-02-13 13:09:57,601] ({main} RAMJobStore.java[initialize]:155) - RAMJobStore initialized.
 INFO [2018-02-13 13:09:57,602] ({main} QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

 INFO [2018-02-13 13:09:57,602] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
 INFO [2018-02-13 13:09:57,603] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1
 INFO [2018-02-13 13:09:57,603] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
 WARN [2018-02-13 13:09:57,605] ({main} ContextHandler.java[log]:2062) - unavailable
javax.servlet.ServletException: Resource class org.apache.zeppelin.server.ZeppelinServer can not be instantiated due to InvocationTargetException
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createSingletonInstance(CXFNonSpringJaxrsServlet.java:396)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createApplicationInstance(CXFNonSpringJaxrsServlet.java:454)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createServerFromApplication(CXFNonSpringJaxrsServlet.java:432)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:93)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
 WARN [2018-02-13 13:09:58,343] ({main} WebAppContext.java[doStart]:514) - Failed startup of context o.e.j.w.WebAppContext@4ddced80{/,file:/opt/mapr/zeppelin/zeppelin-0.7.2/webapps/webapp/,STARTING}{/opt/mapr/zeppelin/zeppelin-0.7.2/zeppelin-web-0.7.2.war}
javax.servlet.ServletException: Resource class org.apache.zeppelin.server.ZeppelinServer can not be instantiated due to InvocationTargetException
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createSingletonInstance(CXFNonSpringJaxrsServlet.java:396)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createApplicationInstance(CXFNonSpringJaxrsServlet.java:454)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createServerFromApplication(CXFNonSpringJaxrsServlet.java:432)
        at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:93)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
 INFO [2018-02-13 13:09:58,411] ({main} AbstractConnector.java[doStart]:266) - Started ServerConnector@8b492c0{SSL-HTTP/1.1}{0.0.0.0:9995}
 INFO [2018-02-13 13:09:58,411] ({main} Server.java[doStart]:379) - Started @6168ms
 INFO [2018-02-13 13:09:58,411] ({main} ZeppelinServer.java[main]:194) - Done, zeppelin server started

 

 

Please advise - we are evaluating mapr and i have a show and tell meeting with stakeholders tomorrow

Outcomes