Graylog conditions and alerts: Limit to windows of time?


(Bronius Motekaitis) #1

Graylog Alerts are really proving their worth here!

How can I limit stream input or condition or alert such that I am notified when a given process or activity being monitored fails (stops producing messages) only during business hours, say, 8a-5p? I won’t even try to tackle “and not school holidays,” unless you have some ideas there…

I have increasingly set these up as heartbeat alert for system processes that must be running (waiting) (and producing log messages) during business hours. After hours, I expect these processes to not produce messages, bc there is no user activity to trigger it.

Thanks
-Bronius


#2

With pipeline you can set a field eg. business_hour true or false, and set your stream based on this field. (Or you can put the message into a stream with pipeline)
And if you have too much time, you can also create eg. a lookup table for special days, and make a decision based on the ours, the weekday and the lookup table.


(Bronius Motekaitis) #3

Ok that is clever!
Hmm… But once a message is marked, I still can’t tell Conditions to anticipate getting dead air (no inbound logs outside business hours). Conditions can only route message, not silence, right? What am I overlooking here?


(Ben van Staveren) #4

You use the business_hours field as part of the condition, or rather, the query in the condition so you can include business hours - e.g. append AND business_hours:"yes" or something to it. Then your condition will be equal to something like “alert if less than X messages and it’s during business hours”.


(Bronius Motekaitis) #5

Thanks @benvanstaveren - I’ll let my grey matter churn on this one: I am still not seeing past how I can give a condition a pipeline-calculated value business_hours:"yes" in a period during which no logs are expected to stream in:
Normal Condition during business hours: Specific messages streaming in, pipeline-decorated with business_hours:yes.
Normal Condition during off-business hours: Specific messages stopped streaming in.
Alarm Condition: Specific messages stopped streaming in (and it is business hours, but there are no messages to decorate)

I appreciate your creative solution and hope my mind can grok what you’re handing.


(Jesse Hills) #6

Hi @texas-bronius

You might find the Aggregates plugin useful for creating an alert for this purpose…

With the above plugin and the pipeline rules that have been discussed above, you should be able to create an alert condition to suit your needs.

Thanks.


#7

I would do two different stream As you wrote the two different needed


(system) closed #8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.