AnsweredAssumed Answered

Drill - Query parquet that is being appended by Spark?

Question asked by john.humphreys on Jan 9, 2018
Latest reply on Jan 10, 2018 by john.humphreys

If I have a spark streaming job that is appending records to a parquet file throughout the day (e.g. 1 parquet file is made per day), can drill query that file throughout the day?


I'd like to save a file a day but I need drill to be able to query the current data as well as the old data.

Outcomes