AnsweredAssumed Answered

FluentD configuration error - invalid time format to parse Hue log files

Question asked by hillbilly on Aug 18, 2017
Latest reply on Sep 5, 2017 by maprcommunity

Hi all,

 

TL;DR   Is there a known bug in the mapr-fluentd RPM package??

 

Our "lab" cluster (5 nodes, RedHat 7.3 on OpenStack) was installed by a MapR consultant, using Ansible playbooks. The monitoring components -- FluentD, ElasticSearch etc. -- have been set up, with the default configuration AFAIK.

 

Installed Packages
mapr-core.x86_64        5.2.1.42646.GA-1        @maprtech
mapr-fluentd.x86_64     0.14.00.201704051222-1  @maprecosystem
...

 

 

A few weeks later I noticed that on the node running Hue, the FluentD log was abnormally large, with piles of entries like this one:

/opt/mapr/fluentd/fluentd-0.14.00/var/log/fluentd/fluentd.log

...

2017-08-18 08:58:47 -0400 [warn]: "[18/Aug/2017 14:58:47 +0200] resource     DEBUG    GET Got response: {\"apps\":null}" error="invalid time format: value = 18/Aug/2017 14:58:47 +0200, error_class = ArgumentError, error = invalid strptime format - `%d/%b/%Y %R:%M:%S %Z'"

...

 

And indeed there is no such code as %R in the Python reference for strptime; that looks like a typo. The correct configuration would be, logically...

 

/opt/mapr/fluentd/fluentd-0.14.00/etc/fluentd/fluentd.conf

...

# hue log
...
  time_format "%d/%b/%Y %H:%M:%S %Z"
  path /opt/mapr/hue/hue-*/logs/runcpserver.log
...

# hue stdout log

...
  time_format "%a %b %d %H:%M:%S %Z %Y"
  path /opt/mapr/hue/hue-*/logs/hue-mapr-runcpserver-*.out
...

 

---------------

Has anyone ever met (and fixed) that issue??

Outcomes