AnsweredAssumed Answered

streaming mapreduce on sandbox,streaming mapreduce in sandbox

Question asked by dkatz on May 21, 2015
I tried running this null-op example from the MapR website in the latest MapR Sandbox for MapR, but the job failed:

[root@maprdemo jobs]# HADOOP="/usr/bin/hadoop"

HADOOP="/usr/bin/hadoop"

[root@maprdemo jobs]# HADOOPSTREAMING="$HADOOP jar /opt/mapr/hadoop/hadoop-0.20.2/contrib/streaming/hadoop-0.20.2-dev-streaming.jar"

HADOOPSTREAMING="$HADOOP jar /opt/mapr/hadoop/hadoop-0.20.2/contrib/streaming/hadoop-0.20.2-dev-streaming.jar"

[root@maprdemo jobs]# $HADOOPSTREAMING -input file:///etc/passwd -output streamOut0 -mapper '/bin/cat' -reducer '/bin/cat'

$HADOOPSTREAMING -input file:///etc/passwd -output streamOut0 -mapper '/bin/cat' -reducer '/bin/cat'

15/05/21 04:54:44 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.j\

obtracker.address

15/05/21 04:54:45 INFO client.MapRZKBasedRMFailoverProxyProvider: Updated RM address to maprdemo/192.168.40.\

159:8032

15/05/21 04:54:45 INFO client.MapRZKBasedRMFailoverProxyProvider: Updated RM address to maprdemo/192.168.40.\

159:8032

15/05/21 04:54:45 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job \

or Job#setJar(String).

15/05/21 04:54:45 INFO mapred.FileInputFormat: Total input paths to process : 1

15/05/21 04:54:45 INFO mapreduce.JobSubmitter: number of splits:2

15/05/21 04:54:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1432155908097_0006

15/05/21 04:54:45 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resource\

s.

15/05/21 04:54:45 INFO security.ExternalTokenManagerFactory: Initialized external token manager class - com.\

mapr.hadoop.yarn.security.MapRTicketManager

15/05/21 04:54:45 INFO impl.YarnClientImpl: Submitted application application_1432155908097_0006

15/05/21 04:54:45 INFO mapreduce.Job: The url to track the job: http://maprdemo:8088/proxy/application_14321\

55908097_0006/

15/05/21 04:54:45 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-root/mapred/local]

15/05/21 04:54:45 INFO streaming.StreamJob: Running job: job_1432155908097_0006

15/05/21 04:54:45 INFO streaming.StreamJob: Job running in-process (local Hadoop)

15/05/21 04:54:46 INFO streaming.StreamJob: map 0% reduce 0%

15/05/21 04:55:18 INFO streaming.StreamJob: map 100% reduce 100%

15/05/21 04:55:19 INFO streaming.StreamJob: Job running in-process (local Hadoop)

15/05/21 04:55:19 ERROR streaming.StreamJob: Job not successful. Error: Task failed task_1432155908097_0006_\

m_000000

Job failed as tasks failed. failedMaps:1 failedReduces:0

15/05/21 04:55:19 INFO streaming.StreamJob: killJob...

15/05/21 04:55:19 INFO impl.YarnClientImpl: Killed application application_1432155908097_0006


Do I need a newer version of the contributed streaming-mapreduce?
Any help much appreciated.

Outcomes