We are currently using Graylog v2.0.3 and I have defined the following alert condition:
Alert is triggered when there are more than 20 messages in the last 5 minutes. Grace period: 1 minute. Including last message in alert notification.
I am getting alerts when there are more than 20 messages counted in the last 5 minutes.
I am currently looking for some ‘back to normal’ alert so that in case there are less than 20 messages counted after 5 minutes I would expect to get an alert saying ‘number of messages counted in the last 5 minutes is for example 15 although the threshold is 20’ so I know it is all good again.
this is a topic that I’m about to open an Github issue about.
I’ll post the link to this here.
As a little workaround, you could define a second alert condition matching your “normal” values and make sure it is stateful. That will fire a notification if your values return to normal, but will always have an alert in the Alert tab.
Thank you for your prompt response.
I appreciate your workaround but in this case I will receive an alert each 5 minutes saying that the status is normal.
We have an integration with Slack and currently looking for getting only 1 alert upon ‘back to normal’.
We know that we are good unless we get an alert saying that number of messages raised above 20 and just want to get 1 alert 5 minutes after the previous alert saying that everything is OK.
This expectation is to get an alert saying that we are back to normal (another option will be to wait for 5 minutes and see that no new alert was triggered).
This feature exist in Sensu and we got used to it before moving to Graylog.
Graylog got (the now optional) stateful alerts. This means, Graylog will only send one alert notification if an alert has been triggered and will then wait until the alert has been resolved plus the grace period for it to start checking the condition again.
