AnsweredAssumed Answered

Balancer not running Any way to force it?

Question asked by jerdavis2 on Mar 22, 2015
Latest reply on Jul 23, 2015 by nani
Hi, I have a very unbalanced cluster, and I've tried adjusting the balancer settings in the UI (and command line) every which way, but the balancer doesn't run. Is there any way to force it, or a process I can restart to hopefully fix this?


    [jdavis@c10-n004 ~]$ maprcli config load -json | grep balancer
       "cldb.balancer.disk.delta":"0",
       "cldb.balancer.disk.max.switches.in.nodes.percentage":"27",
       "cldb.balancer.disk.paused":"0",
       "cldb.balancer.disk.sleep.interval.sec":"120",
       "cldb.balancer.disk.threshold.percentage":"15",
       "cldb.balancer.logging":"0",
       "cldb.balancer.role.max.switches.in.nodes.percentage":"28",
       "cldb.balancer.role.paused":"0",
       "cldb.balancer.role.skip.container.active.sec":"360",
       "cldb.balancer.role.sleep.interval.sec":"120",
       "cldb.balancer.startup.interval.sec":"120",

    [jdavis@c10-n004 ~]$ maprcli dump balancerinfo
    ip:port             percentage  fsid                 usedMB   spid                              capacityMB  fullnessLevel  inTransitMB  outTransitMB 
    10.100.0.111:5660-  98          8453356121283537686  1783452  01a8a20356e25c0700506b1cc60ee2e2  1805764     AboveAverage   0            0            
    10.100.0.102:5660-  77          4576017635451631788  2790870  08dba3772522ed37005487ccb806d2ef  3652500     Average        42622        0            
    10.100.0.118:5660-  75          3297282182380626167  4084857  131df3d0f291c0f90051718e150d8984  5486265     Average        41631        0            
    10.100.0.102:5660-  96          4576017635451631788  5250340  15227afedf17e5130054be663b0450da  5483412     AboveAverage   16557        0            
    10.100.0.127:5660-  85          1951660042482715087  7002040  1e27542cf2960a7e005491baab0d783d  8234348     Average        53640        0            
    10.100.0.113:5660-  19          3487009020909844912  703705   1f41364ce9d896880054e8ed3b0118d1  3655353     UnderUsed      633          0            
    10.100.0.117:5660-  97          4356958765057676754  1752383  22daeddb3e3780ef00540f24340ec960  1824441     AboveAverage   21061        0            
    10.100.0.117:5660-  81          4356958765057676754  2172894  264e07c9fdc6f505005185254203d411  2740143     Average        58872        0            
    10.100.0.100:5660-  84          8121896241193526404  3050556  2851221856312b2f0051965eda09e90b  3643176     Average        15535        0            
    10.100.0.112:5660-  76          45387639755306927    4134246  28e62b48cea96b1600508073d80ca37f  5483412     Average        38697        0            
    10.100.0.115:5660-  97          2006877674427724646  3562689  2a5d07b1c55ab4100052600cb60ec7b2  3652500     AboveAverage   0            0            
    10.100.0.118:5660-  79          3297282182380626167  4333107  2b42184891882f130051718e120c61ed  5482169     Average        46445        0            
    10.100.0.110:5660-  88          6176915784727138298  3201571  2babc50d3ff79d30005040563e01300c  3614653     Average        15460        0            
    10.100.0.111:5660-  98          8453356121283537686  3557757  30c4a18c6cdd721600504646f304ac6d  3617506     AboveAverage   0            0            
    10.100.0.111:5660-  97          8453356121283537686  5298074  33675891f3f994aa00504646f00bd3c8  5429741     AboveAverage   12318        0            
    10.100.0.117:5660-  84          4356958765057676754  1509390  37aee03ea98d9e7a00540f240d068a81  1824441     Average        29231        0            
    10.100.0.126:5660-  71          845400421267898334   3884188  3a260834803d1e390054250d2c0e671a  5488231     Average        41827        0            
    10.100.0.118:5660-  78          3297282182380626167  4323864  3bb094db81ccd13d0051718e14057b61  5486265     Average        545          0            
    10.100.0.104:5660-  81          2277664052127614385  4432784  3bbc8b5ba7be3cb90054cbba330486cb  5483412     Average        28063        0            
    10.100.0.104:5660-  97          2277664052127614385  1775136  3dfedad1a1c098240052056ad208c6f1  1821588     AboveAverage   2            0            
    10.100.0.111:5660-  98          8453356121283537686  5350609  44a709e91e25a99900504646f20108dc  5429741     AboveAverage   0            0            
    10.100.0.101:5660-  97          6304049507233513734  1768921  46ea0e7c82dda8fa00528bcc6906a784  1821588     AboveAverage   155          0            
    10.100.0.103:5660-  13          2534435700298732531  719956   4be74a0c65ff8a04004f8753f5060f6f  5483412     UnderUsed      1014         0            
    10.100.0.113:5660-  98          3487009020909844912  1802291  4e93dacabcb964250054e8ed610dd6aa  1824441     AboveAverage   65           0            
    10.100.0.101:5660-  97          6304049507233513734  1765526  5058ef34b2cc27e90053fb558d05b0f7  1821588     AboveAverage   3835         0            
    10.100.0.103:5660-  98          2534435700298732531  3600789  52f557f45e4a72590054ff87860cf29a  3652500     AboveAverage   0            0            
    10.100.0.102:5660-  97          4576017635451631788  1780466  57e397315f05ec150052056a950ca2fd  1821588     AboveAverage   916          0            
    10.100.0.116:5660-  72          2826232177342589342  2650359  583f6f63b8cc6a090051f6d2830572b3  3655353     Average        1            0            
    10.100.0.110:5660-  23          6176915784727138298  1254670  5b567f81dad96bdf005040563b064ae3  5426887     UnderUsed      15312        0            
    10.100.0.111:5660-  97          8453356121283537686  215587   5bb66856219defbf0053c0485405d4a8  221595      AboveAverage   0            0            
    10.100.0.126:5660-  85          845400421267898334   4658239  5e4b3d46fa4b01440054250d2509b405  5488231     Average        9177         0            
    10.100.0.117:5660-  98          4356958765057676754  1800977  5f8b1652a4447d820054e8ee6c0bed09  1824441     AboveAverage   0            0            
    10.100.0.100:5660-  85          8121896241193526404  4641594  611baa6c9c59774c00518815730563d6  5474088     Average        27255        0            
    10.100.0.115:5660-  96          2006877674427724646  5252363  666dac31de63c653005197cfd80afd1a  5483412     AboveAverage   20518        0            
    10.100.0.101:5660-  79          6304049507233513734  1445520  6c734face6fba0af0054dc09140b37aa  1821588     Average        4912         0            
    10.100.0.104:5660-  98          2277664052127614385  3579475  6d78be32f07d540900520568d00b119a  3652500     AboveAverage   0            0            
    10.100.0.115:5660-  85          2006877674427724646  4620129  713231b316fc4670005197cfd60eb750  5483412     Average        43916        0            
    10.100.0.119:5660-  78          1388303423990164804  4246663  722b91c3d2469e3f0054834b5c06e19e  5488231     Average        42841        0            
    10.100.0.100:5660-  98          8121896241193526404  3580582  790d84b12aea18f60054ff6915040144  3643176     AboveAverage   0            0            
    10.100.0.117:5660-  98          4356958765057676754  3594117  7bfbd613f8f5c62d0054e8ee600a22f8  3655353     AboveAverage   15689        0            
    10.100.0.110:5660-  16          6176915784727138298  288019   7ce03a7953f98e78005112ec4005701d  1802910     UnderUsed      15395        0            
    10.100.0.117:5660-  98          4356958765057676754  3600246  7d72177d30ca4c0a0054be7ffd0d9a2c  3655353     AboveAverage   1            0            
    10.100.0.128:5660-  98          6269351913368737901  8074241  7fb26f0c05dbb7eb005490881304fc97  8235337     AboveAverage   346          0            
    10.100.0.114:5660-  82          7240705629854861643  4520466  80cedd1d4cac610000516f1c4d0e6eaf  5479316     Average        23136        0            
    10.100.0.127:5660-  74          1951660042482715087  6069037  8a4bee1403cd14d9005491bab10249f9  8234348     Average        28848        0            
    10.100.0.119:5660-  97          1388303423990164804  5360227  8d14043e6b0ef9650054187810016709  5488231     AboveAverage   172          0            
    10.100.0.118:5660-  95          3297282182380626167  5217534  8d351580097b6f470051718e110249dd  5486265     AboveAverage   8328         0            
    10.100.0.100:5660-  83          8121896241193526404  4558963  8ebe86ea945ef5bb004f8cc3730e003c  5474088     Average        29335        0            
    10.100.0.114:5660-  78          7240705629854861643  4279986  9234f3e02d45c56a00516f1c5301e964  5483412     Average        28839        0            
    10.100.0.104:5660-  82          2277664052127614385  4498916  938a11e58d97f4e4004f87588903feeb  5479316     Average        44051        0            
    10.100.0.101:5660-  97          6304049507233513734  1772527  947f7bd7effe054b0054189ae60ca92f  1821588     AboveAverage   0            0            
    10.100.0.116:5660-  97          2826232177342589342  3563312  97033e0329e58a7e00519278960bc699  3655353     AboveAverage   983          0            
    10.100.0.103:5660-  11          2534435700298732531  614665   97213aae9877eb31004f8753f2095bc4  5483412     UnderUsed      15088        0            
    10.100.0.100:5660-  85          8121896241193526404  3102934  98c9b1ae2cfd49af00520580590f1476  3643176     Average        19147        0            
    10.100.0.110:5660-  23          6176915784727138298  1238490  9940bb6e4ee09b2a005040563c0aba06  5426887     UnderUsed      15810        0            
    10.100.0.115:5660-  97          2006877674427724646  1755629  9bdf0fe0211a14ab0052793ae00e6be9  1821588     AboveAverage   13214        0            
    10.100.0.112:5660-  79          45387639755306927    4363737  9cbedbe1d9856b4a00508073d70a9d4c  5483412     Average        13800        0            
    10.100.0.128:5660-  72          6269351913368737901  5966696  a41a1b1b9a44c6d600549088150e112b  8235337     Average        43823        0            
    10.100.0.126:5660-  73          845400421267898334   4010582  a578974c523c0ef80054250d2a07b710  5488231     Average        28005        0            
    10.100.0.102:5660-  97          4576017635451631788  1766161  a8948ddd616f23bf005113e59a085298  1821588     AboveAverage   7642         0            
    10.100.0.101:5660-  98          6304049507233513734  1791540  ae33547cef2af66a0054dc093c0e043b  1821588     AboveAverage   513          0            
    10.100.0.113:5660-  98          3487009020909844912  3613921  b3835e36ee2490120054ff89a20cfa4d  3655353     AboveAverage   0            0            
    10.100.0.112:5660-  97          45387639755306927    5341584  b50e6ffecfb01b0400508073d6078cbd  5483412     AboveAverage   15814        0            
    10.100.0.128:5660-  80          6269351913368737901  6610010  c55c807b7ce4f69c005490880e0226c2  8235337     Average        15650        0            
    10.100.0.113:5660-  98          3487009020909844912  3607176  c5f572d1f2f3bbc70054e8ed2d06d392  3655353     AboveAverage   0            0            
    10.100.0.117:5660-  27          4356958765057676754  509562   c7da1ac1132b1bb20054e8ee5004e5f1  1824441     UnderUsed      0            0            
    10.100.0.116:5660-  98          2826232177342589342  1791224  cb4679dbe385b73c0052045f4f044f90  1824441     AboveAverage   518          0            
    10.100.0.127:5660-  75          1951660042482715087  6185200  ccd33250f090f94f005491baae08081d  8234348     Average        13625        0            
    10.100.0.128:5660-  85          6269351913368737901  7049898  d0576dfb49a9cb3200549088100ad538  8235337     Average        30241        0            
    10.100.0.119:5660-  86          1388303423990164804  4691777  d0f6b1771336ad87005418781207be0e  5488231     Average        28435        0            
    10.100.0.103:5660-  98          2534435700298732531  5375340  d0f876505733b601004f8753f4011b50  5483412     AboveAverage   0            0            
    10.100.0.104:5660-  73          2277664052127614385  4006992  d84667d424338c41004f87588d00a0c9  5483412     Average        32547        0            
    10.100.0.127:5660-  68          1951660042482715087  5632596  d955d57316670ffc005491bab30b658d  8234348     BelowAverage   29120        0            
    10.100.0.116:5660-  77          2826232177342589342  4231567  dace38e5fe4d689c00517197780410e2  5486265     Average        27909        0            
    10.100.0.116:5660-  84          2826232177342589342  4605365  dd17e94251b22bcc00519276a10731b0  5482169     Average        13252        0            
    10.100.0.111:5660-  85          8453356121283537686  4586255  e23fe9b05f4f983400504646ef07f73b  5429741     Average        31167        0            
    10.100.0.126:5660-  83          845400421267898334   4542201  e8a3b09bfac4b8250054250d28006f93  5488231     Average        15940        0            
    10.100.0.110:5660-  90          6176915784727138298  4894619  ea47adf415f7d4d5005040563a00f09c  5426887     AboveAverage   0            0            
    10.100.0.112:5660-  83          45387639755306927    4539914  ec2244178426028800508073d506d373  5479316     Average        18153        0            
    10.100.0.117:5660-  75          4356958765057676754  1324346  ef662010111825b70054be6b0b0ba39f  1824441     Average        55837        0            
    10.100.0.119:5660-  77          1388303423990164804  4221815  efcc15e6b6c4853400541878140e27a0  5488231     Average        27133        0            
    10.100.0.114:5660-  73          7240705629854861643  4016184  f59fbdc4744adfc800516f1c5105aef6  5483412     Average        21162        0            
    10.100.0.101:5660-  85          6304049507233513734  1565355  f63367e819c3570100540e2610038db9  1821588     Average        633          0            
    10.100.0.101:5660-  68          6304049507233513734  5001947  f73c5cde324b47ac0054d05b9b044ed6  7314324     BelowAverage   21327        0            
    10.100.0.115:5660-  79          2006877674427724646  4343067  f8c62d0061edfa8d005197cfda03b96e  5483412     Average        40620        0            
    10.100.0.114:5660-  81          7240705629854861643  4458202  f9ae526988af9e9a00516f1c4f0ade84  5483412     Average        31423        0            
    
    No active container moves



Outcomes