AnsweredAssumed Answered

Occasional 'IOException: Stream closed' error running Hive query

Question asked by sridhar.sudanagunta on Sep 15, 2017
Latest reply on Feb 22, 2018 by maprcommunity

Hao Zhu

 

Hi Hao,

 

We are frequently getting the following error for Hive query. Please let me know if you can able to help on this. ?

 

Occasional 'IOException: Stream closed' error running Hive query

 

"Approximately 1 in 200 Hive queries resulting in error "java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. java.io.IOException: Stream closed". Did not see error before most recent MapR upgrade, but happens periodically now.  Usually see it with simple count queries (such as select count(*) form df2_hsr_lakeprd.mbr_pgm_snapshot).  Seems to happen more often early morning (1-6am) than normal business working hours but may just be coincidence.  No file path or handle provided in error message to research on what object the IOException occurred.  Given MapRedTask error as well, assuming it is on a behind-the-scenes file and not a file the tenant has control over.
Incident ticket requested by DevOps to cut down on Talend jobs they are having to rerun.  An error trap is also being developed for our Talend jobs to capture and rerun queries that fail for this error."

Outcomes