AnsweredAssumed Answered

Connecting to drill from spark and getting exception then ConnectionState: Connection timed out for connection string <zookeeper quorum> (MapR 4.0.1)

Question asked by ravi.papisetti on Mar 31, 2016
Latest reply on Feb 6, 2017 by cathy

I am trying to connect to drill from spark application and getting below exception. However it worked fine when it is run individually from eclipse (without any spark related code, just jdbc connection, simple select query). After including spark context and submit spark job to cluster, it is getting connection time out error. Any thoughts? We are using Marp 4.0.1, Drill 1.4, Java 1.7.

 

16/03/31 12:17:38 ERROR ConnectionState: Connection timed out for connection string (zk quorum) and timeout (5000) / elapsed (39283)
oadd.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
        at oadd.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197)
        at oadd.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
        at oadd.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
        at oadd.org.apache.curator.utils.EnsurePath$InitialHelper$1.call(EnsurePath.java:148)
        at oadd.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
        at oadd.org.apache.curator.utils.EnsurePath$InitialHelper.ensure(EnsurePath.java:141)
        at oadd.org.apache.curator.utils.EnsurePath.ensure(EnsurePath.java:99)
        at oadd.org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:74)
        at oadd.org.apache.curator.framework.imps.NamespaceImpl.newNamespaceAwareEnsurePath(NamespaceImpl.java:87)
        at oadd.org.apache.curator.framework.imps.CuratorFrameworkImpl.newNamespaceAwareEnsurePath(CuratorFrameworkImpl.java:457)
        at oadd.org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:224)
        at oadd.org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:183)
        at oadd.org.apache.curator.x.discovery.details.ServiceCacheImpl.<init>(ServiceCacheImpl.java:65)
        at oadd.org.apache.curator.x.discovery.details.ServiceCacheBuilderImpl.build(ServiceCacheBuilderImpl.java:47)
        at oadd.org.apache.drill.exec.coord.zk.ZKClusterCoordinator.<init>(ZKClusterCoordinator.java:104)
        at oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:197)
        at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:151)
        at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:64)
        at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
        at oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
        at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:215)
        at com.cisco.csf.sdaf.catalog.profiler.drill.DrillJDBCConnection.getDrillConnection(DrillJDBCConnection.java:27)
        at com.cisco.csf.sdaf.catalog.profiler.drill.DrillHiveDataProfiler.sdafCatalogProfile(DrillHiveDataProfiler.java:164)
        at com.cisco.csf.sdaf.catalog.profiler.drill.DrillHiveDataProfiler.main(DrillHiveDataProfiler.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:441)

 

Thank you for your help.

Outcomes