Backlog and alerting via mail

Graylog Central (peer support)
1

Hi graylog team,

since i’m using “backlog” i have some issue with alerting via mail. I receive everytime 3 (same)mails.

my Condition :

grafik
grafik.png588×741 28.3 KB

Alert :
********** graylog **********
Alert Description: {check_result.resultDescription} Date: {check_result.triggeredAt}
Alert Condition Title: {alertCondition.title} {if stream_url}Stream URL: {stream_url}{end}
{if backlog} Last messages accounting for this alert: {foreach backlog message}
Source: {message.source} {end}{else}<No backlog> {end}

Mail :
first mail :
********** graylog ********** Alert Description: Stream received messages matching message:“Error” (Current grace time: 0 minutes)
Date: 2019-09-01T18:46:28.942Z
Alert Condition Title: Error
Stream URL: https://graylog.example.com/streams/000000000000000000000001/messages?rangetype=absolute&from=2019-09-01T18:41:28.942Z&to=2019-09-01T18:46:28.942Z&q=*

Last messages accounting for this alert:
Message:Error: limit exceededSource:www.mydomain.com

second mail :

********** graylog ********** Alert Description: Stream received messages matching message:“Error” (Current grace time: 0 minutes)
Date: 2019-09-01T18:46:28.942Z
Alert Condition Title: Error
Stream URL: https://graylog.example.com/streams/000000000000000000000001/messages?rangetype=absolute&from=2019-09-01T18:41:28.942Z&to=2019-09-01T18:46:28.942Z&q=*

Last messages accounting for this alert:
Message:Error: limit exceededSource:www.mydomain.com

third mail :

******* graylog ********** Alert Description: Stream received messages matching message:“Error” (Current grace time: 0 minutes)
Date: 2019-09-01T18:46:28.942Z
Alert Condition Title: Error
Stream URL: https://graylog.example.com/streams/000000000000000000000001/messages?rangetype=absolute&from=2019-09-01T18:41:28.942Z&to=2019-09-01T18:46:28.942Z&q=*

Last messages accounting for this alert:
Source:www.mydomain.com

I would be really grateful if you could answer my question.

2

have you checked if the recipient is a Mailinglist or an alias that duplicates the message?

3

Hi @jan ,

thanks for your quick answer, there is only one mail as recipient for this alarm. Is there any way to debug sending mail for each alert or any logfiles which shows, how many mails and when the mails were sent ? Where are the path of alert and condition on server ?

4

all logs are in one location, that is on most installations /var/log/graylog/server.log (but we wrote down most default locations in the docs: http://docs.graylog.org/en/3.1/pages/configuration/file_location.html ).

If you can’t find a notice about the mail you could raise the logging in Graylog to info or similar. The other option would be - check the logfile of your mailserver.

5

@jan

i couldn’t find any related logs in debug mode and on /var/log/graylog/server.log. The Logs of Mailserver shows that each time was 3x mails sent.
Could you please review my backlog config again and check if it’s 100% ok and give some idee to find the issue ? As i mentioned the issue is since i’m using backlog.

Thanks.

6

E-Mail is the notification - so the reason might be in the condition.

I’m not able to debug that and give a reason for you. It is very likely that the reason is in your configuration.

Check if you might have copied the condition multiple times, or the have multiple notifications.

7

@jan

i’m using 3 differents Conditions :

    exception (Field Content Alert Condition)
    Alerting on stream All messages
Message Backlog : 1

    outofmemory (Field Content Alert Condition)
        Alerting on stream All messages
Message Backlog : 1

    Error (Field Content Alert Condition)
        Alerting on stream All messages
Message Backlog : 1

and 3 Notifications with same names “exception, outofmemory, Error” .

i assume that there is copies of (multiple) condition which is not shown on graylog GUI but it exist on server. Where can i find this conditions / Notifications on server ?

What’s the next step to find the issue ?

We’re going to use graylog on more 200x servers and it’s very important for us to solve the issue.

8

Did you have 3 conditions and 3 notifications on the same stream?

the connection between condition and notification is the stream. That means ONE condition is true, all Notifications for that stream are fired…

If you do not want that - update Graylog to 3.1 to the new alert and notification system.

1 Like
9

Hi @jan

yes, it’s correct. should i have only ONE notification for ALL 3 conditions?

should i remove all 3 notifications after update to 3.1 and create only ONE for all 3 (old) conditions ?

10

As I have written:

the connection between condition and notification is the stream. That means ONE condition is true, all Notifications for that stream are fired…

The alerting in 3.1 is different. When you update, you will notice that.

11

@jan

thanks for your tip.
Best regards.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.