Events using aggregation group by field removes data from notification

SalC · March 19, 2024, 5:15pm

I’ve got an event with an associated notification (Slack). I am filtering for messages containing a tag I added via a Pipeline. I want to only receive a single event if multiple messages are picked up by the filter, so I am setting an aggregation using “Group by Field(s)”. This works like a charm - however, the associated notification seems to be having data stripped from it.

Example:

I am aggregating on the field:
winlogbeat_winlog_event_data_TargetUserName

I am setting event fields as follows:

Field Name	Is Key?	Value Source	Data Type	Configuration

event_id	No	Template	string	        template: "${source.winlogbeat_winlog_event_id}"require_values: false

hostname	No	Template	string		template: "${source.winlogbeat_host_name}"require_values: false

ipaddress	No	Template	string		template: "${source.winlogbeat_winlog_event_data_IpAddress}"require_values: false

service_name	No	Template	string		template: "${source.winlogbeat_winlog_event_data_ServiceName}"require_values: false

task		No	Template	string		template: "${source.winlogbeat_winlog_task}"require_values: false

username	No	Template	string		template: "${source.winlogbeat_winlog_event_data_TargetUserName}"require_values: false

My notification is set as:

--- [Event] --------------------------------------
Title:                ${event_definition_title}
Timestamp:  ${event.timestamp}
Username:    ${event.fields.username}
Hostname:    ${event.fields.hostname}
IP Address:   ${event.fields.ipaddress}
Task:		   ${event.fields.task}
Event ID:         ${event.fields.event_id}

${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}

${end}${end}

In the actual notification though, the values for Hostname, IP Address, Task, and Event ID are stripped, see example below:

Alert EventName triggered:

EventDescription
— [Event] --------------------------------------
Title: Event Title
Timestamp: 2024-03-19T15:40:14.691Z
Username: username_here
Hostname:
IP Address:
Task:
Event ID:
— [Backlog] ------------------------------------
Last messages accounting for this alert:

Setting any/some/all of the event fields to be a key, has no noticeable effect.

Any ideas? Thanks in advance!

gsmith · March 20, 2024, 4:26am

Hey @SalC

Since those messages are coming in from winlogbeat it change the field/s naming convention.
either you can disable that or edit the template to something like this.

--- [Event] --------------------------------------
Title:                ${event_definition_title}
Timestamp:  ${event.timestamp}
Username:    ${event.fields.username}
Hostname:    ${event.fields.hostname}
IP Address:   ${event.fields.ipaddress}
Task:		   ${event.fields.task}
Event ID:         ${event.fields.event_id}

${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}
Source:            ${message.source}
EventID:           ${message.fields.winlogbeat_winlog_event_id}
HostName:          ${message.fields.winlogbeat_host_name}
IP Address:        ${message.fields.winlogbeat_winlog_event_data_IpAddress}
TargetUserName:    ${message.fields.winlogbeat_winlog_event_data_TargetUserName}
Logon Type:        ${message.fields.LogonType}
${end}${end}

SalC · March 20, 2024, 12:41pm

@gsmith Thanks for the reply! I copy/pasted the notification as you re-wrote it, but unfortunately the fields are still empty – the backlog has the info however?

— [Event] --------------------------------------
Title: AD Domain Admin: Multiple Failed Logins
Timestamp: 2024-03-20T12:37:15.740Z
Username: username_here
Hostname:
IP Address:
Task:
Event ID:
— [Backlog] ------------------------------------
Last messages accounting for this alert:
Source: dc1
EventID: 4771
HostName: dc1.contoso.com
IP Address: ::ffff:169.254.123.111
TargetUserName: username_here
Logon Type:

gsmith · March 21, 2024, 4:59am

Hey @SalC

Ok so the message fields work.

As for your Event section try this. The thing is those are events

--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}

I did something like this a while back. If they don’t show , that mean the naming configuration is incorrect.

SalC · March 22, 2024, 5:04pm

@gsmith That might work, but then the notification won’t have the data I want?

I changed it to:

--- [Event] --------------------------------------
Title:                ${event_definition_title}
Timestamp:  ${event.timestamp}
Username:    ${event.username}
Hostname:    ${event.hostname}
IP Address:   ${event.ipaddress}
Task:		   ${event.task}
Event ID:         ${event.event_id}

and as expected, only get this:

— [Event] --------------------------------------
Title: AD Domain Admin: Successful Login
Timestamp: 2024-03-22T16:55:14.691Z
Username:
Hostname:
IP Address:
Task:
Event ID:

I can’t figure out why the data is being stripped when I do it like I originally posted because it works if I don’t aggregate the notifications?

gsmith · March 23, 2024, 3:58am

Hey @SalC

You need to separate EVENT and Message Fields.

If you want to extract a field from your message you need to add the following under Fields section. The SOMEFIELD needs to match what is in the message your try to get. EVENT’s are another entity.

Example:

SomeName: ${message.fields.SOMEFIELD}

Here is an example from that post I link above.

This is just an example you need to modify it for your environment.

-- [Event Definition] ---------------------------
Title:       ${event_definition_title}
Description: ${event_definition_description}
Type:        ${event_definition_type}
--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}
Fields:
${foreach event.fields field}  ${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
Messages; ${foreach backlog message}
TargetUserName: ${message.fields.TargetUserName}
WorkstationName: ${message.fields.WorkstationName}
EventReceivedTime: ${message.fields.EventReceivedTime}
Source; ${message.fields.source}
${end}
${end}

EDIT Here is a another example i made a while back.

SalC · March 25, 2024, 9:30pm

@gsmith Thanks for working through this with me! So, if I’m understanding correctly, to get the data from winlogbeat into the event, instead of defining this within the event definition itself as an event field, ex:

Field Name	Is Key?	Value Source	Data Type	Configuration	Actions
event_id	No	Template	string	template: “${source.winlogbeat_winlog_event_id}”require_values: false

I need to do something like this within the notification?
event_id: ${message.fields.winlogbeat_winlog_event_id}

?

Thanks!
-Sal

gsmith · March 26, 2024, 12:21am

Hey @SalC

You got it

Just like I showed you above…

The Event Definition defines a Filter where some of the search arguments are to express in terms of parameters.

These instructions can be found below.

https://go2docs.graylog.org/5-0/interacting_with_your_log_data/notifications.html

https://go2docs.graylog.org/5-0/interacting_with_your_log_data/alerts_and_events.html

SalC · March 26, 2024, 4:22pm

@gsmith Thanks! I’ve been following this docs but some of the info you’ve shared wasn’t there, so I’m learning new things as we go.

The notification still comes in empty, as follows:

Title: AD Domain Admin: Successful Login
Timestamp: 2024-03-26T17:23:13.845Z
Username:
Hostname:
IP:
Service Name:
Task:
Event Id:

Here’s my notification definition now:

--- [Event] --------------------------------------
Title: ${event_definition_title}
Timestamp: ${event.timestamp}
Username: ${event.fields.username}
Hostname: ${message.fields.winlogbeat_host_name}
IP: ${message.fields.winlogbeat_winlog_event_data_IpAddress}
Service Name: ${message.fields.winlogbeat_winlog_event_data_ServiceName}
Task: ${message.fields.winlogbeat_winlog_task}
Event Id: ${message.fields.winlogbeat_winlog_event_id}

gsmith · March 26, 2024, 9:15pm

Hey @SalC

Think your missing part of the template configuration. I seen in the previous post you had it working.

SalC · March 27, 2024, 3:07pm

@gsmith Thanks for working with me on this, and sorry if I’m taking a bit to figure stuff out… but here’s the thing, which I think might be a bug or I’m doing something wrong…

This works perfectly for the backlog (like you pointed out) … but for the event, it’s blank?

As an example:

--- [Event] --------------------------------------
Title: ${event_definition_title}
Timestamp: ${event.timestamp}
Username: ${event.fields.username}
Hostname: ${message.fields.winlogbeat_host_name}
IP: ${message.fields.winlogbeat_winlog_event_data_IpAddress}
Service Name: ${message.fields.winlogbeat_winlog_event_data_ServiceName}
Task: ${message.fields.winlogbeat_winlog_task}
Event Id: ${message.fields.winlogbeat_winlog_event_id}

--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}
Source:            ${message.source}
EventID:           ${message.fields.winlogbeat_winlog_event_id}
HostName:          ${message.fields.winlogbeat_host_name}
IP Address:        ${message.fields.winlogbeat_winlog_event_data_IpAddress}
TargetUserName:    ${message.fields.winlogbeat_winlog_event_data_TargetUserName}
Logon Type:        ${message.fields.LogonType}
${end}${end}

If I stitch this together from previous posts, this would result in something like this:

Title: AD Domain Admin: Successful Login
Timestamp: 2024-03-26T17:23:13.845Z
Username:
Hostname:
IP:
Service Name:
Task:
Event Id:

— [Backlog] ------------------------------------
Last messages accounting for this alert:
Source: dc1
EventID: 4771
HostName: dc1.contoso.com
IP Address: ::ffff:169.254.123.111
TargetUserName: username_here
Logon Type:

gsmith · March 27, 2024, 9:39pm

Hey @SalC

This is not a bug, its about understanding the concept of Graylogs Email Template.

I think you are almost there, maybe I can explain it better.

In the Email Template you would have sections.

Event Section

If you look closely you’ll notice the macro’s used to get the BASIC information from your Event Definition.

--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}

Message Fields Section

This section is probably what you want to configure. This will add or subtract the information you want when you send a notification. I believe this already works for you since you posted this above.

Fields:
${foreach event.fields field}  ${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
Messages:            ${foreach backlog message}
TargetUserName:      ${message.fields.TargetUserName}
WorkstationName:     ${message.fields.WorkstationName}
EventReceivedTime:   ${message.fields.EventReceivedTime}
Source:              ${message.fields.source}
${end}
${end}

Results:
If you put them together it should look like this. Noticed the macro’s used in each section.

-- [Event Definition] ---------------------------
Title:       ${event_definition_title}
Description: ${event_definition_description}
Type:        ${event_definition_type}
--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
Key:                  ${event.key}
Priority:             ${event.priority}
Alert:                ${event.alert}
Timestamp Processing: ${event.timestamp}
Timerange Start:      ${event.timerange_start}
Timerange End:        ${event.timerange_end}
Fields:
${foreach event.fields field}  ${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
Messages:           ${foreach backlog message}
TargetUserName:     ${message.fields.TargetUserName}
WorkstationName:    ${message.fields.WorkstationName}
EventReceivedTime:  ${message.fields.EventReceivedTime}
Source:             ${message.fields.source}
${end}
${end}

In this forum there are hundreds of post about email template and really good examples, for instance this one.

Hope that helps

system · April 10, 2024, 9:40pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Events using aggregation of a threshold as a trigger missing event fields data in Notification? Graylog Central (peer support)	2	131	July 12, 2023
Graylog 3.1 Alert Notification using Event Definition with Aggregation Graylog Central (peer support)	15	2477	January 5, 2020
Custom fields in Event Definition Graylog Central (peer support)	2	1699	March 3, 2021
ALERT Event Group By with multiple fields and email backlog Graylog Central (peer support)	3	1705	May 30, 2021
Alert and events systems from v 3.1 Graylog Central (peer support)	5	989	September 10, 2019

Events using aggregation group by field removes data from notification

Related Topics