Hello everyone, I’m writing this post due to an issue in log processing on Graylog.
1. Describe your incident:
The logs are not showing in real-time over Graylog and because of that we are not receiving Slack notifications. This occurs only (for now) with the AWS Cloudtrail logs, we have 3 accounts and 1 SQS for each account and the inputs in Graylog were created successfully (no problems in configuration reviewing that). We notice in SQS the messages accumulated are rounding 8000 and the “Message in flight” are a small amount when we refreshing many times.
In Graylog, we saw the messages incoming against an event in AWS are having a delay between 10 and 15 minutes (Example, I make a “DescribeSecurityGroup” and that message arrives on Graylog then 10 minutes, not in Real-time).
2. Describe your environment:
OS Information: (EC2 c5) Ubuntu 22.04 LTS, 4 cores, 8 GB RAM, 1 disk (20 GB) and 1 partition (49 GB).
Package Version: Graylog 4.3.3 Noir, Elasticsearch 7.10.2, MongoDB 6.0.0 (NOTE: the 3 are installed in the same server).
Service logs, configurations, and environment variables:
3. What steps have you already taken to try and solve the problem?
We were trying changing any configuration of Graylog and/or Elasticsearch about JVM (from 1 to 2 or 3 GB) but that’s not working. Then, we tried modifications on Índices options but I’m not sure what is the more optimized configuration for an installation in we use case.
NOTE: I’m not understanding clearly the documentation about Índices and Pipelines on Graylog (I think maybe the problem can be fixed implementing something about these), if anybody can give a little additional explanation about that I appreciate so much.