Hi community,
this is the only thing that make me unhappy everyday.
there I activate the “Remove matches from ‘All messages’ stream” now can’t find the logs from last night!
I have also changed the input status from global to local but still after business hours they get stopped.
How can I fix this permanently?
Make sure that your Graylog nodes have enough resources (CPU, memory, IOPS) and that there are no excessive stream rules (usually some complex regular expressions) which slow down processing.
You can also check the Graylog metrics (see System/Nodes/Metrics) for stream rules with a high execution time (org.graylog2.plugin.streams.StreamRule.{id}.executionTime).
I have increased the resources but still one of my most important streams with following rules it switch off automatic itself.
Field message must match regular expression .*tcpflags="TCP SYN ACK".* (TCP-SYN Denial-of-Service Attack)
Field message must match regular expression .*tcpflags="TCP ACK".*
Field message must match regular expression .*TCP PSH.*
Field message must match regular expression .*TCP URG.*
Field message must match regular expression .*TCP RST.* (Session Denial&Abort)