AnsweredAssumed Answered

configuring hadoop.log.dir on EMR

Question asked by sorenmacbeth on Jun 5, 2013
Latest reply on Jun 6, 2013 by gera

I'm running into problem running M3 version 2.1.2 on EMR. My root partitions are filling up. The main culprit are the logs under the `hadoop.log.dir` directory set in `taskcontrollers.cfg`. I would like to configure this to point to a different directory.

What is the recommended way to do with as part of my cluster startup? I'm assuming I'm going to have to write a custom bootstrap action. My first attempt was to use sed to change the directory using sed and then restart the tasktrackers via the `maprcli` using the following shell script:

set -x
sudo sed -i 's/hadoop.log.dir=.*/hadoop.log.dir=\/var\/tmp/' $TASKCONTROLLER_CFG
$MAPCLI node services -filter [ip=="*"] -tasktracker restart

This didn't work. I got:

<code>ERROR (10002) -  Service: tasktracker is not configured on node: [nodename]</code>

How should I do this properly?