AnsweredAssumed Answered

cachemgr detected a hang

Question asked by stormcrow on May 1, 2013
Latest reply on May 7, 2013 by nabeel
mfs has crashed on one of our compute nodes twice in the last 24 hours. I'm not sure of the cause. Is this log snippet from mfs.log cause for alarm?

    2013-05-01 16:53:18,7579 INFO  fileserver.cc:7265 x.x.0.0:0 Sending vol list with 12 volumes.
    2013-05-01 16:54:31,5017 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 79441 ms
    2013-05-01 16:54:44,7052 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 92637 ms
    2013-05-01 16:55:08,4546 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 107180 ms
    2013-05-01 16:55:14,6451 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 97751 ms
    2013-05-01 16:55:17,3607 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 116845 ms
    2013-05-01 16:55:17,3607 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 116735 ms
    2013-05-01 16:55:41,8829 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 8 took 70380 ms
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:0 totalDirty:1473 maxDirty: 253174 reserved:0 numWaits:0 nInLru:545559 start:1 end 632935
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:1 totalDirty:5 maxDirty: 434012 reserved:0 numWaits:0 nInLru:482230 start:934333 end 1416568
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:2 totalDirty:431418 maxDirty: 434012 reserved:2560 numWaits:495885 nInLru:1165158 start:1416569 end 3013977
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:3 totalDirty:5 maxDirty: 120558 reserved:0 numWaits:0 nInLru:301392 start:632936 end 934332
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:4 totalDirty:0 maxDirty: 0 reserved:0 numWaits:0 nInLru:0 start:0 end 0
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:5 totalDirty:3349 maxDirty: 4286545 reserved:0 numWaits:0 nInLru:10710835 start:1 end 10716364
    2013-05-01 16:59:03,3290 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:6 totalDirty:0 maxDirty: 2079646 reserved:0 numWaits:0 nInLru:2025531 start:1 end 2079645
    2013-05-01 16:59:03,3290 ERROR  cachemgr.cc:91 x.x.0.0:0 Hang in CacheMgr: npending 9, Oldest CacheOp 6
    2013-05-01 16:59:03,3291 ERROR  fileserver.cc:6990 x.x.0.0:0 cachemgr detected a hang, mfs is potentially deadlocked. killing self
    2013-05-01 17:00:11,2441 INFO  mapfs.cc:801 x.x.0.0:0

Outcomes