Ganglia deployment in MapR v5.0

Document created by bpark on Feb 1, 2016
Version 1Show Document
  • View in full screen mode

Author: Bo soon Park

 

Original Publication Date: October 7, 2015

 

Environment :     MapR v5.0

                              Ganglia 3.7

 

Issue :

Deploying the ganglia in the MapR v5.0, Ganglia can't collect the cldb/fileserver metric from MapRIn the cldb.log, you can find out the error message like this :

2015-10-05 20:49:15,444 WARN Alarms [RPC-8]: Alarm raised: NODE_ALARM_SERVICE_NFS_DOWN:bonode03:NODE_ALARM; Cluster: my2.cluster.com; Node: bonode03; Message: Can not determine if service: nfs is r
unning. Check logs at: /opt/mapr/logs/nfsserver.log
Exception in thread "Timer thread for monitoring cldb" java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets        
     at org.apache.hadoop.metrics.ganglia.GangliaContext.xdr_string(GangliaContext.java:246)        
     at com.mapr.fs.cldb.counters.MapRGangliaContext31.emitMetric(MapRGangliaContext31.java:99)        
     at org.apache.hadoop.metrics.ganglia.GangliaContext.emitRecord(GangliaContext.java:169)       
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.emitRecords(AbstractMetricsContext.java:319)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.timerEvent(AbstractMetricsContext.java:305)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.access$000(AbstractMetricsContext.java:57)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext$1.run(AbstractMetricsContext.java:264)        
     at java.util.TimerThread.mainLoop(Timer.java:555)        
     at java.util.TimerThread.run(Timer.java:505)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.Charsets        
     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)        
     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)        
     at java.security.AccessController.doPrivileged(Native Method)        
     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)        
     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)        
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)        
     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)         ... 9 more
Exception in thread "Timer thread for monitoring fileserver" java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets        
     at org.apache.hadoop.metrics.ganglia.GangliaContext.xdr_string(GangliaContext.java:246)        
     at com.mapr.fs.cldb.counters.MapRGangliaContext31.emitMetric(MapRGangliaContext31.java:99)        
     at org.apache.hadoop.metrics.ganglia.GangliaContext.emitRecord(GangliaContext.java:169)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.emitRecords(AbstractMetricsContext.java:319)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.timerEvent(AbstractMetricsContext.java:305)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext.access$000(AbstractMetricsContext.java:57)        
     at org.apache.hadoop.metrics.spi.AbstractMetricsContext$1.run(AbstractMetricsContext.java:264)        
     at java.util.TimerThread.mainLoop(Timer.java:555)        
     at java.util.TimerThread.run(Timer.java:505)
2015-10-05 20:50:47,127 INFO ZKDataRetrieval [Thread-2-EventThread]: Process path: /services_config/cldb/bonode03. Event state: SyncConnected. Event type: NodeDataChanged

Root Cause:

There is no commons-io-2.4.jar file in /opt/mapr/lib.

 

Workaround:

Please make the symbolic link and restart cldb and ganglia like this :

#ln -s /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-io-2.4.jar /opt/mapr/lib/commons-io-2.4.jar
#maprcli node services -filter [csvc==cldb] -cldb restart
#/etc/init.d/gmond restart
#/etc/init.d/gmetad restart

If you can't see the cldb/fileserver metrics in the ganglia, please contact MapR support with reference Bug ID 19734.

 

Attachments

    Outcomes