AnsweredAssumed Answered

hive ignoring chunk-size?

Question asked by thurman on May 15, 2012
Latest reply on May 15, 2012 by thurman
We're attempting to modify the chunk-size on hive data. There is no issue at all doing this via `hadoop mfs`, however when we use hive to import the data into an already chunk-size modified directory it still writes it as the default 256.
<p>
me@gw:~# hadoop mfs -setchunksize 65536000 /user/hive/warehouse/mydata/ <br>
me@gw:~# cat /mapr/my.cluster.com/user/hive/warehouse/mydata/.dfs_attributes<br>
lines beginning with # are treated as comments<br>
Compression=true<br>
ChunkSize=65536000<br>
</p>
And now import process via hive, then check file's chunk-size

me@gw:~# hadoop mfs -ls /user/hive/warehouse/mydata/<br>
Found 1 items<br>
-rwxrwxrwx Z   3 mydata users  581955679 2012-05-15 18:41  268435456 /user/hive/warehouse/mydata/mydata.tsv<br>


Is there some value specific to hive that is needed for hive to respect .dfs_attributes values?
Or maybe something else we are over looking?

Outcomes