AnsweredAssumed Answered

Drill 1.10 Memory Leak with CTAS + partition by

Question asked by sgudavalli on Feb 14, 2018
Latest reply on Feb 15, 2018 by cathy

Hello,

 

I am having an issue with CTAS used along with PARTITION BY.

I see there is memory leaked by query. Is this a known Bug? I am on Drill 1.10.0.201707281514

 

0: jdbc:drill:schema=dfs> CREATE TABLE by_yr (yr, ngram, occurrances) PARTITION BY (yr) AS SELECT columns[1] yr, columns[0] ngram, columns[2] occurrances FROM dfs.`/tmp/googlebooks-eng-all-5gram-20120701-zo.tsv` LIMIT 100;
Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (1114112)
Allocator(op:0:0:0:EasySubScan) 1000000/1114112/1114112/10000000000 (res/actual/peak/limit)


Fragment 0:0

[Error Id: fe4e06d5-fdd8-4789-8a06-e99dc0b2566c on bossrnd113060:31010] (state=,code=0)
0: jdbc:drill:schema=dfs> CREATE TABLE by_yr (yr, ngram, occurrances) AS SELECT columns[1] yr, columns[0] ngram, columns[2] occurrances FROM dfs.`/tmp/googlebooks-eng-all-5gram-20120701-zo.tsv` LIMIT 100;
+-----------+----------------------------+
| Fragment  | Number of records written  |
+-----------+----------------------------+
| 0_0       | 100                        |
+-----------+----------------------------+
1 row selected (0.221 seconds)
0: jdbc:drill:schema=dfs>

 

 

Please help, if I can fix this memory leak by any configuration changes.

 

Regards

Shiv

Outcomes