[Solved] Numeric field values being rounded in graphs

Graylog Central (peer support)
1

I’m experiencing extremely odd behavior from the graph and quick values output for a particular field across two different inputs.

  • Numeric values for this field are being rounded instead of displayed as a float in graphs & quick values
  • The actual field value is displayed as the expected float in the message output
  • This started occurring at approximately midnight UTC after a new index was started in elastic search
  • This is occurring on the load_avg_1m field across 2 different inputs
  • This field value is being extracted via a Add Key=Value pairs as fields converter
  • The field value is being converted via a numeric converter on the extracted field
  • Both inputs also have load_avg_5m and load_avg_15m fields which are extracted and converted in the same manner and work as expected

Example showing values of all three load_avg fields with only load_avg_1m graphing incorrectly:

graph-output
graph-output.png1070×1402 338 KB

This issue is rendering a few of our dashboards essentially useless as the graphs no longer show the actual load on the monitored instances.

Any help or advice would be appreciated.

2

To me it looks like ES returns only part of the search results. I have had the same. You can try relaxing ES search timeouts, and look if it helps. Anyway; when the dashboard widget refreshes automatically, these missing values might jump in and make the graph smooth (if this is the problem in your case).

3

Sadly that’s not the issue. The dips-to-zero in the graph are unwanted, but expected as the Heroku log drain provides data minute by minute, but not necessarily ON every minute. You’ll notice in the histogram that there’s a good block of one message per minute, and around the gaps, the following or interim minutes have two messages.

The issue the topic is about relates to the differences between the blue line, whose data is being rounded to the nearest 0.5 and the red and green lines whose data is being displayed as it is stored.

4

Please create a bug report on GitHub if you think you’ve discovered a bug.

5

Probably a bug, but a transient one and most likely elastic search related. Once the index rolled over last night at 00:00 UTC the values started displaying correctly in the graphs & quick values as well as in the field text.

Possibly related to receiving some aberrant value at some point in that index, but all the values that display show up as the expected numeric value. Marking this solved and chalking it up to code ghosts.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.