AnsweredAssumed Answered

Execute Drill Query From Spark?

Question asked by john.humphreys on Feb 28, 2018
Latest reply on Feb 28, 2018 by john.humphreys

There seem to be a nearly infinite amount of class-path related issues when you try to use the Apache Drill driver in Spark.  E.g. you get errors with java.lang.NoClassDefFoundError: oadd/org/apache/log4j/Logger when you try to create a connection.


Given MapR heavily supports Spark and Drill, do you have an example of calling a simple drill query in Spark?

 

Note that I'm just trying to call a metadata refresh on new parquet files once they're written in spark, so I don't even need it integrated with any spark features; I literally just need to run a single simple statement as a one-off and am having trouble.

 

Drill 1.12 driver, Spark 2.1.0.


  <groupId>
org.apache.drill.exec</groupId>
  <artifactId>drill-jdbc-all</artifactId>
  <version>1.12.0</version>

Outcomes