Email Alert sending separate emails for different messages

[peaky10]

Using the Graylog version: 3.3.2

I am using the below template for Graylog alerts and have not been receiving emails as expected.

--- [Event Definition] ---------------------------
Title:       ${event_definition_title}
Description: ${event_definition_description}
Type:        ${event_definition_type}
--- [Messages] -----------------------------------
${if backlog} ${foreach backlog message}
Response Code: ${message.fields.response_code} | HTTP_REFERRER: ${message.fields.http_referrer}
${end}${else}No Backlog${end}
--- [Event] --------------------------------------
Timestamp:            ${event.timestamp}
Message:              ${event.message}
Source:               ${event.source}
--------------------------------------------------

While receiving email, I’m getting separate email for every message in backlog as below:

First Email:

--- [Event Definition] ---------------------------
Title:       http referrer
Description: Http referrer with status code !=200
Type:        aggregation-v1
--- [Messages] -----------------------------------

Response Code: 404 | HTTP_REFERRER: https://xxxx.xxx.xxxx/xxx

--- [Event] --------------------------------------
Timestamp:            2020-07-28T08:31:48.859Z
Message:              http referrer
Source:               e9ac8be01bcb
--------------------------------------------------

Second Email:

--- [Event Definition] ---------------------------
Title:       http referrer
Description: Http referrer with status code !=200
Type:        aggregation-v1
--- [Messages] -----------------------------------

Response Code: 502 | HTTP_REFERRER: https://xxxxx.xxxx/xxxx

--- [Event] --------------------------------------
Timestamp:            2020-07-28T09:01:05.465Z
Message:              http referrer
Source:               e9ac8be01bcb
--------------------------------------------------

Third Email:

--- [Event Definition] ---------------------------
Title:       http referrer
Description: Http referrer with status code !=200
Type:        aggregation-v1
--- [Messages] -----------------------------------

Response Code: 404 | HTTP_REFERRER: https://xxx.xxx.xxx.xxx/xxx

--- [Event] --------------------------------------
Timestamp:            2020-07-28T09:41:17.301Z
Message:              http referrer
Source:               e9ac8be01bcb
--------------------------------------------------

and so on.

I have been getting thousands of emails like this below:

I want to get all the backlog messages in a single email body, like below:

--- [Event Definition] ---------------------------
Title:       http referrer
Description: Http referrer with status code !=200
Type:        aggregation-v1
--- [Messages] -----------------------------------

Response Code: 404 | HTTP_REFERRER: https://xxxx.xxx.xxxx/xxx
Response Code: 502 | HTTP_REFERRER: https://xxxxx.xxxx/xxxx
Response Code: 404 | HTTP_REFERRER: https://xxx.xxx.xxx.xxx/xxx

--- [Event] --------------------------------------
Timestamp:            2020-07-28T08:31:48.859Z
Message:              http referrer
Source:               e9ac8be01bcb
--------------------------------------------------

Kindly guide me with the solution. Thanks!

[jan (Jan Doberstein)]

how did you configure the events and alerst?

[peaky10]

Took the screenshot of Summary:

I hope it helps in understanding the problem.

[peaky10]

Hello @jan,
Please help anything I can try to make this work?

Thanks!

[marick]

@peaky10

s below:

Guys thank you for great job!

I have the same problem. One email for every message.
Graylog server 3.3.3
elasticsearch 6.8.11
Oracle linux 7.8

[marick]

How many Events will Filter & Aggregation create?

The Filter & Aggregation Condition will generate different number of Events, depending on how it is configured:

• Filter: One Event per message matching the filter
• Aggregation without groups: One Event every time the aggregation result satisfies the condition
• Aggregation with groups: One Event per group whose aggregation result satisfies the condition

So I grouped by field ( beats_type): Example: count(beats_type) > 1
If I group by message I get the following error:

“Fielddata is disabled on text fields by default. Set fielddata=true on [full_message] in order to load fielddata in memory by uninverting the inverted index. Note
that this can however use significant memory. Alternatively use a keyword field instead., errorDetails=[Fielddata is disabled on text fields by default. Set fiel
ddata=true on [full_message] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternativ
ely use a keyword field instead.]}
at org.graylog2.indexer.cluster.jest.JestUtils.specificException(JestUtils.java:110)
at org.graylog.plugins.views.search.elasticsearch.ElasticsearchBackend.doRun(ElasticsearchBackend.java:263)
at org.graylog.plugins.views.search.elasticsearch.ElasticsearchBackend.doRun(ElasticsearchBackend.java:74)”

Now I receive 1 mail for 10 messages if they come in 5 seconds interval.
I hope I’m doing it correctly
Hope it helps!