AnsweredAssumed Answered

Significant delay noticed between subsequent hive queries. Thread end up waiting for 10 minutes Hive.renameFiles() --> MapRFileSystem.rename(line no:486)

Question asked by vvk on Jul 1, 2013
Latest reply on Jul 3, 2013 by gera
While executing subsequent hive queries, I am noticing that there is significant delay (- 13 minutes). During this time, there is no CPU activity, but I do see the network busy.

Can some one throw some light on what could be happening here.


Hive Version: 0.9.0
MapR Version: 2.1.2

**Thread - Stack Trace**

"pool-7-thread-2" prio=10 tid=0x000000006107a000 nid=0x7d98 waiting on condition [0x0000000045be9000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000cf36a560> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1923)
at com.mapr.fs.Inode.getDataIntoCache(Inode.java:783)
at com.mapr.fs.MapRFsInStream.updateReadAhead(MapRFsInStream.java:480)
at com.mapr.fs.MapRFsInStream.dropAndRefillCurrentPage(MapRFsInStream.java:127)
at com.mapr.fs.MapRFsInStream.read(MapRFsInStream.java:163)
- locked <0x00000000cf368218> (a com.mapr.fs.MapRFsInStream)
at java.io.DataInputStream.read(DataInputStream.java:83)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:74)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:227)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:217)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:160)
at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:486)
at org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:2035)
at org.apache.hadoop.hive.ql.metadata.Table.replaceFiles(Table.java:532)
at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1259)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:234)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1333)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1125)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:957)

Outcomes