Datanode/Opensearch Issue After Upgrade to graylog-enterprise 7.0

Graylog Central (peer support)
1

1. Describe your incident:

I’m using graylog security on a Ubuntu 22 VM and until the latest version 6.3.5 everything was working smoothly.

When I initially wanted to upgrade graylog-server to 7.0.1 then graylog-datanode to 7.0.1 the CPU spikes to 100% (no mater how much CPU I’m assinging to the instance).

I revert the upgrade and try to only upgrade graylog-server to 7.0.1 and let graylog-datanode to 6.3.5 and it went better … for approx. 24h, then the CPU spikes to 100% again

2. Describe your environment:

  • OS Information: Ubuntu 22.04 LTS

  • Package Version:
    graylog-datanode: 6.3.5-1
    graylog-enterprise 7.0.1-1
    mongodb-org-server 8.0.15
    Opensearch (as per the folder name): 2.15.0

  • Service logs, configurations, and environment variables:
    datanode.log is full of line like :

[OpensearchProcessImpl] [2025-11-28T15:34:15,865][WARN ][o.o.s.b.SearchBackpressureService] [SERVER] [monitor_only mode] cancelling task [5418249] due to high resource consumption [elapsed time exceeded [30.8s >= 30s]]

3. What steps have you already taken to try and solve the problem?

I tried to avoid updating to datanode 7.0.1 but the issue raise again after 24h

2

Hello @Nono,

This is difficult to diagnose without a holistic view of the cluster. Could you share the full data node log?

3

It turns out that our illuminate Sigma rules had too much to handle after the update so it DDoS itself by timing out and re-applying the rules/search.

After waiting 24/48h, the last upgrade went fine.

1 Like
4

Thanks for the update @Nono.

While it’s great you are happy to rely on the community for answers, as an enterprise user you can also contact Graylog support for help should you require it.

1 Like
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.