AnsweredAssumed Answered

Drill - Un-cache Parquet Metadata

Question asked by john.humphreys on Feb 28, 2018
Latest reply on Feb 28, 2018 by arjunkr

Does anyone know of any possible way to "uncache" and/or disable the metadata generated by running the "refresh table metadata" command on parquet directories in Drill?

 

I'm finding that old metadata literally makes queries fail with "file does not exist" exceptions (when you change the underlying file system by deleting/updating parquet files).  So, using the cache really makes it hard to query files that are being dynamically updated, even if you went out of your way to use maxdir() and versioning to do it properly without a race condition.

Outcomes