AnsweredAssumed Answered

Was the distributed cache object changed?

Question asked by takeshi_nakano on Jul 22, 2011
Latest reply on Jul 22, 2011 by takeshi_nakano
I got an error on using Hive.
Does anyone have advice for this?

    [root@ip-10-148-11-174 ~]# rpm -qa | grep mapr-hive
    mapr-hive-1.0.0.10353GA-0
    mapr-hive-internal-1.0.0.10353GA-0
    [root@ip-10-148-11-174 ~]# /opt/mapr/bin/hive -f test2.hiveql
    Hive history file=/tmp/root/hive_job_log_root_201107131323_2063991988.txt
    Total MapReduce jobs = 2
    Launching Job 1 out of 2
    Number of reduce tasks is set to 0 since there's no reduce operator
    Starting Job = job_201107131318_0002, Tracking URL =
    http://ip-10-148-11-174.ap-northeast-1.compute.internal:50030/jobdetails.jsp?jobid=job_201107131318_0002
    Kill Command = /opt/mapr/hadoop/hadoop-0.20.2/bin/../bin/hadoop job
    -Dmapred.job.tracker=maprfs:/// -kill job_201107131318_0002
    2011-07-13 13:23:29,502 Stage-1 map = 0%, reduce = 0%
    2011-07-13 13:23:32,532 Stage-1 map = 100%, reduce = 100%
    Ended Job = job_201107131318_0002 with errors
    FAILED: Execution Error, return code 2 from
    org.apache.hadoop.hive.ql.exec.MapRedTask
    [root@ip-10-148-11-174 ~]#

On this time, All TaskTracker got error like this and I think this caused this problem.

    Error initializing attempt_201107131318_0002_m_000000_0:
    java.io.IOException: The distributed cache object
    maprfs://
    10.148.11.174:7222/tmp/hive-root/hive_2011-07-13_13-23-19_522_4197750802800709184/-mr-10004/6b559f5b-9c05-4f61-b77d-85c9127241ee#HIVE_PLAN6b559f5b-9c05-4f61-b77d-85c9127241ee
    changed during the job from 7/13/11 1:23 PM to 7/13/11 1:23 PM
    at
    org.apache.hadoop.filecache.TrackerDistributedCacheManager.downloadCacheObject(TrackerDistributedCacheManager.java:513)
    at
    org.apache.hadoop.filecache.TrackerDistributedCacheManager.localizePublicCacheObject(TrackerDistributedCacheManager.java:593)
    at
    org.apache.hadoop.filecache.TrackerDistributedCacheManager.getLocalCache(TrackerDistributedCacheManager.java:201)
    at
    org.apache.hadoop.filecache.TaskDistributedCacheManager.setupCache(TaskDistributedCacheManager.java:183)
    at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1506)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1074)
    at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1497)
    at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1387)
    at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:3535)
    at
    org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:3466)

WHat does this mean? Who changed these files?
"**The distributed cache object maprfs://
10.148.11.174:7222/tmp/hive-root/xxx changed during the job from 7/13/11 1:23 PM to 7/13/11 1:23 PM**"

Regards,

Outcomes