graylog version: v3.3.2+ade4779 VM resources: 16 vCPU + 96GB RAM One node
Recently the dashboard performance experiences downgrade so I tried to fine-tune the settings by increasing the graylog JVM heap space to 16GB and the buffer processors settings in graylog server.conf:
I have tried to stop all inputs except one, disable all pipelines, and delete all extractors created shortly before this happened. But don’t help.
I found out that there are tons of unprocessed messages and the process and output buffer are all exhausted. Maybe this is the reason? And if it is, can anyone suggest a way to alleviate the pain?
when you have added those 16 cores to the VM that hosts Elasticsearch AND Graylog it is very likely that both are fighting for the CPUs …
You should limit the number of cores that are available for Elasticsearch and not overcommit the processors on Graylog.
The picture that you have output buffer filled is the indicator that your elasticsearch is needing to long to index the messages and return/free the output processor of Graylog, what causes a jam in the working queue.
I have done some research on how to optimize the Elasticsearch and adjust some settings including memory and swap file.
The issue is seemingly gone for days now.
But I can’t find where to restrict the processor number for Elasticsearch. I can’t find the setting in Elasticsearch.yml (I am running it as a service). Can only find some running it in Docker or from command line.
