I run Graylog 3.3 in Docker at home on a single server. Sometimes my disk usage gets high on the Docker host, and as such, Elasticsearch goes read-only.
When this happens, I’m able to see the number of unprocessed messages on the
Nodes page (screenshot below).
I’m able to manually set my Elasticsearch cluster back to read-write, and then Graylog starts processing messages again.
Is there a search I can use to alert me when unprocessed messages crossed a specific threshold? For example:
unprocessed_messages > 2000?
I don’t believe there’s anything you can use directly built in, but we do this by monitoring via the API. For uncommitted messages, the following would work:
curl -u api_username -H ‘Accept: application/json’ -X GET ‘https://graylogserver.yourdomain.suffix:9000/api/system/metrics/org.graylog2.journal.entries-uncommitted’
If the above doesn’t work you can try with http or using the --insecure switch.
Hope this helps.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.