AnsweredAssumed Answered

Spark SQL + Thrift Server - Generally Confused

Question asked by john.humphreys on Jun 19, 2017
Latest reply on Jun 21, 2017 by MichaelSegel

I'm looking to have a persistent spark context where I can load data frames and from parquet files and query them interactively.


I've heard of using a thrift server for this, but I'm a little unclear how it works.


I see a small amount of information in the Spark SQL and DataFrames - Spark 2.1.1 Documentation, but it's not overly helpful.  Equally, I see a few blurbs on the IBM site etc. that don't cover much.


I don't see anything about how to actually load up data frames from parquet, cache them, and query them though (or even anything remotely that complex).  


Does anyone know of a tutorial or demo that walks through this, or a place where the information can be learned?  Or can someone explain it here to get me started?