Event definitions not triggering alerts

Graylog Central (peer support)
1

1. Describe your incident:
Twice in the past couple of weeks I have observed a situation where an event definition should generate an alert and corresponding notification but does not. I can take the event definition filter parameters back to the pertinent stream and see that they match, but no alert was triggered. I can recreate the situation in which an alert should be triggered by the event definition, observe the qualifying message populate the stream within the defined search window, and no alert is triggered. If I disable the event definition and re-enable it the event definition will then begin triggering alerts as expected.

Looking at the mongo tables I see that the event definition “alert” field is set to true in the event_definitions collection. I see that the “max_processed_timestamp” field is recent (in line with other event definitions) in the event_processor_state collection.

Has anyone else encountered this?

2. Describe your environment:

  • OS Information:
    Ubuntu 20.04
  • Package Version:
    4.2.7
  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

Looked for obvious misconfigurations in the mongo collections. Resetting event definitions corrects it, but doesn’t inspire much confidence.

4. How can the community help?

Hoping someone else has run into and solved this already.

2

@ttsandrew

How ya doing.
I haven’t yet but now I’m looking incase this is happing and I don’t know about it. I’m also running the same version

Did you post this in Github yet? This seams very odd, more like a bug perhaps. Its like fixing a windows server, turn if off and back on fixes stuff.
I assume there is nothing in the logs either?

3

@ttsandrew

FYI.
If you give me some configurations I can test this out in my lab. If that would help?

1 Like
4

definitely sounds like a bug since toggling it off and on seems to fix it. I’d be curious to see what the event definition and alert looks like to see if it’s something I could replicate also.

5

I have the same problem
We are running v4.2.5

The strange thing is that most of the alerts are triggering, but some does not. Less than 1% is missing that should have triggered.

Is there any way to see a log of when the alert-function have been running, and which timespans they have looked for alerts in.

/Mikael

6

Ok so I updated my Lab Server on 04/04/2022 I receive Alerts that were create but now Under Alerts there is no display
example
Before

image
image1886Ă—452 42.7 KB

Now

image
image1894Ă—408 29.1 KB

EDIT: My issue is resolved. Solution was to rotate the Index Set: Graylog Events
Some reason it stopped displaying under Alerts section. Probably doesn’t pertain to this issue.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.