Graylog: Send a notification containing a list of matching events of a given timespan instead of seperate notifications for each event

[stev-e]

Hi everybody,

1. Describe your question:

I have Graylog server set up as a syslog collector. For testing purposes I achieved the following:
Send a notification whenever there is an match with log level<4 (errors and worse) with the fields:

• source
• message

What I would like to achieve are scheduled searches for matches with log level<4 in a given timespan (every hour for the past last hour) and aggregate all found entries into one single notification that holds all the fields for every match.

Notification content:
match 1:

• source
• message

match 2:

• source
• message
  […]

2. Describe your environment:

• OS Information: Ubuntu latest

• Package Version: latest

• Service logs, configurations, and environment variables: does not apply

3. What steps have you already taken to try and solve the problem?

I read the instructions for Filter&Aggregation a few times now and I got no idea how to achive it. Maybe I am using the wrong terminology to search for suitable answers…

4. How can the community help?

Could you please provide me with a hint what is the correct term/part of the documentation that I should read more carefully?

Thanks in advance!

[gsmith]

Hello && Welcome @stev-e

If you want to do a search every hour or 24 hours that would be in your “Event Deinition” settings. Then match or reconfigure your Notification settings to correspond to it.

Also showing what you have configured would be easier to show what you need to do, just an idea

[stev-e]

Hi @gsmith , thank you for replying I think you’re right, here are some pictures to show, what I have and what I want to achieve.

First of all, the Event definition:

→ Need to split the post into multiple ones to send attachments

[stev-e]

Then the corresponding Notification:

[stev-e]

That leads to a mail like this one:

So you see, I repeat to look for hits in a certain timespan. Those hits are parsed (fields are source and message) and for every single hit a mail is sent.

[stev-e]

---

What I instead would love is: Take all the hits of a timespan and instead of sending an email for each and every hit consolidate it to one mail that contains a list of field entries:

Is this possible?

[gsmith]

Hey @stev-e

Things to look at are in red box

So you have no grace peroid for notification, meaning it will send any and all alerts right away.

“Search Within 15” minutes, meaning past 15 minutes it will search.
“Execute search every 10”, meaning every 10 minutes it will search so it looks like your over lapping your search/s.

You might want to look at this here

[stev-e]

Hi @gsmith , I read a few guides and tried your suggestion. But it is still not what I want. When applying this configuration (with or without ‘Message Backlog = 10’) like here:

If I issue syslog errors on a test machine:

logger -p local0.err "test1"
logger -p local0.err "test2"
logger -p local0.err "test3"

the result now is that I only get one notification (test1).
Before I got three seperate notifications (1. notification: test1; 2. notification: test2; 3. notification: test3).
What I want is one notifaction bundling all three messages (1. notification: test1, test2, test3).

[gsmith]

Hey @stev-e

@stev-e

What I would like to achieve are scheduled searches for matches with log level<4 in a given timespan (every hour for the past last hour)

This statment does not match you configurations shown in that screen shot.

Here is a demo perhaps it might help.

EXAMPLE:

• Search for “Level:<4”
• In stream “all Messages”
• Execute a search Ever 24 hours, THEN search the past 24 hours
• Create Event IF the count is greater then 0 in the stream called “All messages”, ALERT.
• Notification , Grace Period is set to 1 day. So ever DAY send a notification but only Only 10 messages

[stev-e]

Ah, switching to to the “count() > 0” logic made the difference. Now there is a backlog of messages that are reported in one single mail. Thanks so far.