Stream's total incoming messages metric

Graylog Central (peer support)
1

I would like to read streams’ input messages for performance monitoring.
(4 nodes, GL v3.1.2 or 2 nodes with GL v3.2.2)

If I check the browser’s traffic when I check the streams menu, I got an api/cluster/metrics/multiple request (POST), and a json answer with the numbers by node.

If I try to check the API browser, I can ask the previous metric also.
Or I can use api/system/metrics/multiple, but it shows the metric of the current node, not the full system (answer is the same under api/cluster/metric’s by node id.)
And it is tha same if I try to ask from api/system/metrics/org.graylog2.plugin.streams.Stream._ID.incomingMessages

So is there one metric when I can get the sum stream messages count (eg total, or last 1 min avg), or I have to sum it manually after a request?
Or any fast way to request the all messages number in a stream?
I don’t want to overload the system with the requests, but it could be helpful if I want to know who start to send messages storm.

Sometimes I use this, but not for full time monitoring.

time curl -XGET -u $USER 'https://IP/api/search/universal/relative/stats?field=source&query=*&range=300&filter=streams%3A_ID_&pretty=true' 2>/dev/null | jq '.count'
2988975

real    0m1.467s
user    0m0.091s
sys     0m0.066s

But for the last 5 min it takes more then 1 sec (for last day 14 sec). I have more than 50 streams, so it needs about 2 minutes. I don’t want to run it every 5/10 minutes.

2

I did something like this in the past:

export RANGE=360; http GET  https://graylog.local.lan/api/sources?range=$RANGE | jq --argjson r1 "$RANGE" ' .sources | to_entries|map("\(.key)=\(.value/$r1)")|.[]'

It uses httpie and jq and is kind of accurate.

3

thanks, its so fast.
about 0,4s for last 5 mins
But we have too much sources, so it need too much work to monitor it.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.