Performance advice. I'm missing something

having Graylog and ES fight for the same ressources does not make it easier.

With your given ES HEAP Elasticsearch will eat all available RAM - because it takes 31GB from the available 50GB waht left 19GB. Then the OS Filesystemcache of Lucene tries to occupie another 31GB making your memory consumption to -12GB - adding the default 1GB HEAP of Graylog we are at -13GB RAM. Now the OS do also need some RAM. As I did not know how you added the SAN (multipath?) that might eat some additional RAM to handle that all. what will eat over all ~15GB more RAM than you have.

Now to the Cores - you have 8vCPUs, with HT? For a GL only System your configuration should be something like Inputbuffer_processors 2, Processbuffer_processors 4, Outputbuffer_processors 2 - But as ES is very distinct about ressources it SEES in the system available it will try to occupy them all for itself. Unless you restrict it with configuration settings. So you have GL and ES fight for the available cores …

With your given processor configuration in Graylog you have configured that Graylog can process with 75 cores and push with 75 connections the same time up to 5000 messages a second to Elasticsearch.

Where are the ressources that Elasticsearch is able to eat the ingested messages?

2 Likes