Hey thanks for the quick anwser,
I was originally asking in a general sense, if there were maybe just some obvious trick or flag I had to set.
Graylog has 12 cores with 24GB of RAM, 12GB Heap Size for the graylog jvm. The 3 ES nodes have 4 cores and 32GB of RAM each, 16GB ES jvm heap size. None of the machines are hitting the resource limits.
The ~600 events run every 10 seconds searching in the last 10 seconds of 1 stream with its own index set, ~360GB, 12 indices with 3 shards + replicas each.
Overall 1TB in Logs seperated into a few index sets.
We have a few hundred extractors over all the different inputs and a few processing pipelines doing small things, looking at the time all of those take doesn’t look too bad.
Overall it’s performing absolutely perfect! During heavy times it will write 10k-20k messages to ES without issues. Just the events which are somehow processed slowly.