AnsweredAssumed Answered

mfs crash. Cause for concern?

Question asked by stormcrow on Apr 12, 2013
Latest reply on May 3, 2013 by nabeel
One of our cluster nodes (a file and task) sustained an mfs crash today. Is this cause for alarm? What does this log snippet actually mean?

    2013-04-12 13:40:58,6439 INFO  containerrestore.cc:2840 x.x.0.0:0 Updating mirror id 0 on container 4223534421
    2013-04-12 13:41:26,7433 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 69734 ms
    2013-04-12 13:41:26,7577 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 69748 ms
    2013-04-12 13:41:30,0927 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 73079 ms
    2013-04-12 13:41:43,0050 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 83502 ms
    2013-04-12 13:42:04,1737 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 65529 ms
    2013-04-12 13:42:04,1737 INFO  dcleaner-sm.cc:897 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 60100 ms
    2013-04-12 13:42:09,2861 INFO  containerresync.cc:3946 x.x.0.0:0 Freeing the resync slab WAs
    2013-04-12 13:44:12,3657 INFO  dcleaner-sm.cc:970 x.x.0.0:0 parent lock for parent type 3 fifo 0 took 66583 ms
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:0 totalDirty:1272 maxDirty: 253174 reserved:0 numWaits:0 nInLru:545745 start:1 end 632935
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:1 totalDirty:6 maxDirty: 434012 reserved:0 numWaits:0 nInLru:482229 start:934333 end 1416568
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:2 totalDirty:288369 maxDirty: 434012 reserved:512 numWaits:1316336 nInLru:1308816 start:1416569 end 3013977
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:3 totalDirty:6 maxDirty: 120558 reserved:0 numWaits:0 nInLru:301391 start:632936 end 934332
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:4 totalDirty:0 maxDirty: 0 reserved:0 numWaits:0 nInLru:0 start:0 end 0
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:5 totalDirty:3320 maxDirty: 4286545 reserved:0 numWaits:0 nInLru:10711346 start:1 end 10716364
    2013-04-12 13:46:00,9692 INFO  cachemgr.cc:88 x.x.0.0:0 LRU:6 totalDirty:0 maxDirty: 2079646 reserved:0 numWaits:0 nInLru:2040324 start:1 end 2079645
    2013-04-12 13:46:00,9692 ERROR  cachemgr.cc:91 x.x.0.0:0 Hang in CacheMgr: npending 2, Oldest CacheOp 6
    2013-04-12 13:46:00,9693 ERROR  fileserver.cc:6990 x.x.0.0:0 cachemgr detected a hang, mfs is potentially deadlocked. killing self
    2013-04-12 13:47:20,6980 INFO  mapfs.cc:801 x.x.0.0:0

Outcomes