AnsweredAssumed Answered

mapr metrics logging into mysql

Question asked by jacob_shaan on Dec 6, 2012
Latest reply on Dec 7, 2012 by yufeldman
I am running a 4 node mapr cluster with 3 data nodes and the other node runs cldb, metrics, zk, jobtracker and webserver. The last node also runs a mysql server for logging the metrics.

After changing password fields in /opt/mapr/conf/db.conf and /opt/mapr/conf/hibernate.cfg.xml from *** to the actual password, I see all the metrics being logged in the following tables.

    +-----------------------------------+
    | Tables_in_metrics                 |
    +-----------------------------------+
    | JOB                               |
    | JOB_ATTRIBUTES                    |
    | JOB_EVENT                         |
    | JOB_HISTORY_PROCESSING_BATCH      |
    | METRIC_TRANSACTION                |
    | METRIC_TRANSACTION_EVENT          |
    | METRIC_TRANSACTION_SUMMARY_BATCH  |
    | METRIC_TRANSACTION_SUMMARY_DAILY  |
    | METRIC_TRANSACTION_SUMMARY_YEARLY |
    | NODE                              |
    | TASK                              |
    | TASK_ATTEMPT                      |
    | TASK_ATTEMPT_ATTRIBUTES           |
    | TASK_ATTEMPT_EVENT                |
    | TASK_EVENT                        |
    +-----------------------------------+

Now JOB* tables and TASK* tables are self evident. When I check the entries in NODE table, I just see only one entry which is that of the node running cldb, webserver and jobtracker. Why don't the other nodes show up?

Furthermore, when I looked at the METRIC_TRANSACTION* tables, I see the metrics corresponding to the node running cldb, webserver and jobtracker only. Again why don't the metrics of other nodes show up?

Here is the dump of results of some queries over those tables :

    mysql> select * from NODE;
    +------------------------------+------------------------------+------------------+---------------------+---------------------+
    | NODE_ID                      | HOSTNAME                     | HOSTID           | CLUSTER_ID          | CREATED             |
    +------------------------------+------------------------------+------------------+---------------------+---------------------+
    | ip-10-151-30-79.ec2.internal | ip-10-151-30-79.ec2.internal | 484a33393b55120a | 6776384455427329620 | 2012-12-04 23:18:57 |
    +------------------------------+------------------------------+------------------+---------------------+---------------------+
    1 row in set (0.00 sec)
    
    
    mysql> select NODE_ID, count(*) from METRIC_TRANSACTION GROUP BY NODE_ID;
    +------------------------------+----------+
    | NODE_ID                      | count(*) |
    +------------------------------+----------+
    | ip-10-151-30-79.ec2.internal |   427089 |
    +------------------------------+----------+
    1 row in set (0.50 sec)

Any help will be appreciated.


Outcomes