AnsweredAssumed Answered

Querying the filesystem is sometimes quick and sometimes slow

Question asked by oae on Nov 4, 2013
Latest reply on Nov 4, 2013 by shaka
One of our customer's is using mapr-2.1.3.
He is experiencing extreme variability in speed when doing a path lookup to the file-system.

Explanation:
(The code is about querying the filesystem, resolving one glob path with all its files and it block-locations. Basically the same thing the standard FileInputFormat is doing.)

Same path 'maprfs:/a/b/c/20130115/*' which resolved to 96 paths (2.34GB or an average of 24MB per file) is taking 15 min one time and then later on just below one sec.

Any clues what this could be related to ?
Is that some namenode warmup ?
Or any known issues around that ?
Any good ways of debugging that ?

Any help appreciated!
Johannes

Outcomes