Resource Usage >
How do I configure Drill to use MapR local volume for spilling space?
See all drill best practice FAQs.
By default, each Drillbit uses its local disk "/tmp" as the spill directory. However, if the local disk is not large enough for a complex query that requires a lot of spilling space, another choice is to use the MapR local volume as the spill directory. Of course, the disks used by MapR local volume must be large enough.
The location of the spilling area is controlled by the following 2 parameters:
The detailed step by step instructions on how to configure this space is described here:
How to use MapR local volume as the spill directory for Apache Drill | Open Knowledge Base
Is there a performance advantage to using a node local dedicated disk for spill, instead of one managed by MapR-FS? I see in the example that replication is set to 1 for this, but is there still overhead in passing local spill to the FS?
Are there disadvantages to using a local / simple disk?
Performance should be the same since MapR Local Volume will also only use the local disk(s) on specific node.
As long as you have enough disk space for that dedicated disk, it should be fine. The only thing is you have to manage it by yourself.
Typical node configuration in cluster will have 1 or 2 disk for local /tmp, while there are typically 8 or more disks allocated to MapR-FS storage pools. By using the MapR-FS local volume you spill to a large number of disks than the typical local /tmp, which in most cases provides an increase in capacity as well as throughput.
Are local volumes (topology moved) supported in Community Edition? See error in other reply.
Is setting the topology to a local volume not supported with a Community Edition license?
I don't see the ability to change volume topology in the web UI, and using the CLI gives a license error:
[~:$] maprcli volume move -name mapr.nfd001.***.local.spill -topology /data/default-rack/nfd001.***
ERROR (10010) - Volume Move: No license for requested operation
I'm attempting the approach shown in https://community.mapr.com/external-link.jspa?url=http%3A%2F%2Fwww.openkb.info%2F2016%2F04%2Fhow-to-use-mapr-local-volum…
Try to use this command rather
maprcli volume create -name mapr.<node>.local.drillspill -path /var/mapr/local/<node>/drillspill -replication 1 -localvolumehost <node>
You may also want to look at this JIRA
That maprcli command works. Thanks!
Retrieving data ...