#  delay in sending graylog alert

**URL:** <https://community.graylog.org/t/delay-in-sending-graylog-alert/29438>\
**Category:** Graylog Central (peer support)\
**Tags:** alert\
**Created:** [July 5, 2023, 6:07pm UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438 "2023-07-05T18:07:48Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rodrigomanoel](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@rodrigomanoel](https://community.graylog.org/u/rodrigomanoel)\
**Post date:** [July 5, 2023, 6:07pm UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/1 "2023-07-05T18:07:48Z")

</div>

I’m using Graylog version 5.1.0+14ba491 and I’m having a delay in sending alerts, sometimes it arrives a few minutes apart or hours later.  
Has anyone gone through this.

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [July 6, 2023, 4:34am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/2 "2023-07-06T04:34:28Z")

</div>

Hey @rodrigomanoel

I have a long time ago, sometimes its resources, timestamp or a configuration issue.

---

<div class="post-metadata">

**Author:** ![patrickmann](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/patrickmann/32/9091_2.png) [@patrickmann](https://community.graylog.org/u/patrickmann)\
**Post date:** [July 6, 2023, 5:21am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/3 "2023-07-06T05:21:19Z")

</div>

Alerts get delayed when Graylog is falling behind on data processing/indexing.

From an existing issue:

> “everything is search based, so if the data isn’t getting in, presumably because the system is overloaded, it won’t be able to run alerts, because well they wouldn’t find anything. messages like `2022-01-19T12:31:36.150Z DEBUG [EventProcessorExecutionJob] Event processor <palo-deauth/616df603d4846155e9b57a55> couldn't be executed because of a failed precondition (retry in 5000 ms)` mean that graylog knows it’s behind on data processing/indexing, so it will delay running the alert job until it’s reasonably sure it has processed data for the time range the alert is supposed to run.”

---

<div class="post-metadata">

**Author:** ![patrickmann](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/patrickmann/32/9091_2.png) [@patrickmann](https://community.graylog.org/u/patrickmann)\
**Post date:** [July 6, 2023, 5:37am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/4 "2023-07-06T05:37:08Z")

</div>

There was also an issue with alerts not being processed when message rate is very low. The fix is already in 5.1 though, so that doesn’t apply to your case.

> <https://github.com/Graylog2/graylog2-server/pull/13556>
>
> \## Description
> 
> In our current implementation of the alert system, there is a …check that is responsible for ensuring that all nodes have indexed their messages by a certain time. Only if this check is successful, we create events. However, there are two scenarios in which the alert system does not create events in time. 
> 
> \- The first one is when we have multiple nodes and the message rate decreases. In this case we always select the post indexing time of the node with the oldest post indexing time and use it for the previously mentioned check. The danger here is that one node not receiving messages for some time would be enough to prevent the alert system from creating events, since the post-indexing time of that node would always be used at a low message rate.
> 
> \- The second scenario is when a message arrives that leads to an event, and no other message arrives for some time afterwards. In this case, the event would be created too late. Like in this issue #13228. The following image illustrates this. 
> !\[AlertSystem\](https://user-images.githubusercontent.com/25727579/192558047-81b34505-f970-44e1-9588-fa819d7ecec9.png)
> 
> 
> This PR fixes the above two cases, but uses a different approach. The general approach is to determine if there are busy nodes in the cluster, and if there are, wait for them to index the messages until a certain time.
> 
> close #6770
> 
> /jenkins-pr-deps Graylog2/graylog-plugin-enterprise#4194

---

<div class="post-metadata">

**Author:** ![rodrigomanoel](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@rodrigomanoel](https://community.graylog.org/u/rodrigomanoel)\
**Post date:** [July 6, 2023, 11:50am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/5 "2023-07-06T11:50:04Z")

</div>

I increased the memory of my graylog to see if the problem solves it and I’m waiting for the alerts to be sent.  
My version is 5.1 but it is already asking to update again.  
Do you want me to put some configuration file here?

---

<div class="post-metadata">

**Author:** ![patrickmann](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/patrickmann/32/9091_2.png) [@patrickmann](https://community.graylog.org/u/patrickmann)\
**Post date:** [July 6, 2023, 1:37pm UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/6 "2023-07-06T13:37:07Z")

</div>

Are you ingesting messages? At what rate?  
Do you see any messages in the log that might be related?

---

<div class="post-metadata">

**Author:** ![rodrigomanoel](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@rodrigomanoel](https://community.graylog.org/u/rodrigomanoel)\
**Post date:** [July 11, 2023, 7:04pm UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/7 "2023-07-11T19:04:53Z")

</div>

I am getting this error message

 ![Capturar](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/7/7517076459ba88976dde09abe8f7857b1a56e095.jpeg)

---

<div class="post-metadata">

**Author:** ![patrickmann](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/patrickmann/32/9091_2.png) [@patrickmann](https://community.graylog.org/u/patrickmann)\
**Post date:** [July 13, 2023, 6:18am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/8 "2023-07-13T06:18:32Z")

</div>

Is this related to the delayed alerts? If not, please start a new topic.

Also, to get useful responses please provide as much information as possible on your setup, what you have already tried, etc. A generic error message is generally not enough to be able to provide any useful suggestions.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [July 27, 2023, 6:19am UTC](https://community.graylog.org/t/delay-in-sending-graylog-alert/29438/9 "2023-07-27T06:19:09Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
