August 21, 2018, 12:25pm
We’re currently using the
Datadog metrics reporter. It’s working fine but we wanted to filter the number of metrics it sends to Graylog. I tried doing that by adding the following line to Graylog config:
metrics_datadog_include_metrics = org.graylog2.shared.journal.KafkaJournal.uncommittedMessages
However, when I added this line and restarted the Graylog server, the filtering did not kick in. All Graylog metrics were sent to Datadog, while I expected only
uncommittedMessages to be sent. I tried this one one more host with the same result.
I’ve only 2 other metrics related configuration:
metrics_datadog_enabled = true
metrics_datadog_transport = UDP
I am using the follwing JAR which does seem to contain the code required for metric filtering:
Any input on what could be going wrong is greatly appreciated.
August 21, 2018, 12:46pm
the metric you are looking for is
org.graylog2.journal.entries-uncommitted on each node to get the nodes journal.
August 21, 2018, 2:22pm
@jan I see the
org.graylog2.shared.journal.KafkaJournal.uncommittedMessages metric being reported to Datadog, and it’s defined here in code as well:
this.throttleThresholdPercentage = intRange(throttleThresholdPercentage, 0, 100);
this.serverStatus = serverStatus;
this.maxSegmentSize = segmentSize.toBytes();
// Max message size should not be bigger than max segment size.
this.maxMessageSize = Ints.saturatedCast(maxSegmentSize);
this.writtenMessages = metricRegistry.meter(name(this.getClass(), "writtenMessages"));
this.readMessages = metricRegistry.meter(name(this.getClass(), "readMessages"));
this.writeDiscardedMessages = metricRegistry.meter(name(this.getClass(), "writeDiscardedMessages"));
registerUncommittedGauge(metricRegistry, name(this.getClass(), "uncommittedMessages"));
// the registerHdrTimer helper doesn't throw on existing metrics
this.writeTime = registerHdrTimer(metricRegistry, name(this.getClass(), "writeTime"));
this.readTime = registerHdrTimer(metricRegistry, name(this.getClass(), "readTime"));
final Map<String, Object> config = ImmutableMap.<String, Object>builder()
// segmentSize: The soft maximum for the size of a segment file in the log
// segmentMs: The soft maximum on the amount of time before a new log segment is rolled
In any case, I just tried using the configuration you suggested, but again, all other metrics are being reported as well. Basically, the filtering does not seem to be working at all.
August 21, 2018, 3:39pm
This might be a bug - does it happen too if you select two values and not just one? Or if you select
Would be nice to get a detailed bug report over at:
August 21, 2018, 4:22pm
Thanks, I’ve posted an issue since I observe the same problem with a filter involving a wildcard too:
September 4, 2018, 4:22pm
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.