AnsweredAssumed Answered

Apache Drill: org.apache.thrift.transport.TTransportException null

Question asked by chris.crawford on Mar 29, 2018
Latest reply on Apr 2, 2018 by shamirwasia
Branched to a new discussion

MapR Team--

When trying to run a Drill query I receive the following error: 

SYSTEM ERROR:
MetaException: Got exception: org.apache.thrift.transport.TTransportException null

 

When I look at the log file (drillbit_queries.json) it shows a remoteAddress as an edge node computer in my network which is not associated with my cluster.  It also shows that Drill tried to submit the query on port 42102.  Any ideas where Drill is picking this information up?

 

The larger error appears to be:

2018-03-29 16:47:29,314 [qtp177048897-57] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: MetaException: Got exception: org.apache.thrift.transport.TTransportException null

 


[Error Id: 88405583-fb4d-4c98-8516-5bf158b5e4e2 on mapr03.wired.carnoustie:31010]
        at org.apache.drill.exec.rpc.AbstractDisposableUserClientConnection.sendResult(AbstractDisposableUserClientConnection.java:85) ~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:809) ~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:870) ~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:293) ~[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
2018-03-29 16:59:01,074 [2542ac6a-66da-82e9-9b49-85a254a65ac0:foreman] INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id 2542ac6a-66da-82e9-9b49-85a254a65ac0: select hostname from sys.drillbits where `current` = true
2018-03-29 16:59:01,101 [2542ac6a-66da-82e9-9b49-85a254a65ac0:foreman] WARN  o.a.d.e.s.h.DrillHiveMetaStoreClient - Failure while attempting to get hive databases. Retries once.
org.apache.hadoop.hive.metastore.api.MetaException: Got exception: org.apache.thrift.transport.TTransportException null
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213) ~[hive-metastore-1.2.0-mapr-json-1710.jar:1.2.0-mapr-json-1710]

 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1045) ~[hive-metastore-1.2.0-mapr-json-1710.jar:1.2.0-mapr-json-1710]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.getDatabasesHelper(DrillHiveMetaStoreClient.java:203) ~[drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$DatabaseLoader.load(DrillHiveMetaStoreClient.java:517) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$DatabaseLoader.load(DrillHiveMetaStoreClient.java:510) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache.get(LocalCache.java:3937) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) [guava-18.0.jar:na]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithAuthzWithCaching.getDatabases(DrillHiveMetaStoreClient.java:423) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getSubSchema(HiveSchemaFactory.java:153) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.<init>(HiveSchemaFactory.java:147) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.registerSchemas(HiveSchemaFactory.java:132) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]

at org.apache.drill.exec.store.hive.HiveStoragePlugin.registerSchemas(HiveStoragePlugin.java:99) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:396) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:110) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:99) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:162) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:151) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:137) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.planner.sql.SqlConverter.<init>(SqlConverter.java:112) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:101) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:991) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:276) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

2018-03-29 16:59:01,122 [2542ac6a-66da-82e9-9b49-85a254a65ac0:foreman] WARN  o.a.d.e.s.h.DrillHiveMetaStoreClient - Failure while attempting to get hive databases. Retries once.
org.apache.hadoop.hive.metastore.api.MetaException: Got exception: org.apache.thrift.transport.TTransportException null
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213) ~[hive-metastore-1.2.0-mapr-json-1710.jar:1.2.0-mapr-json-1710]
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1045) ~[hive-metastore-1.2.0-mapr-json-1710.jar:1.2.0-mapr-json-1710]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.getDatabasesHelper(DrillHiveMetaStoreClient.java:203) ~[drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$DatabaseLoader.load(DrillHiveMetaStoreClient.java:517) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$DatabaseLoader.load(DrillHiveMetaStoreClient.java:510) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache.get(LocalCache.java:3937) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) [guava-18.0.jar:na]
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) [guava-18.0.jar:na]
        at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithAuthzWithCaching.getDatabases(DrillHiveMetaStoreClient.java:423) [drill-storage-hive-core-1.11.0-mapr.jar:1.11.0-mapr]

Outcomes