Here’s roughly my setup:
- 3 * graylog web 4GB 2 Cores 2.90GHz
- 3 * elasticsearch 16GB Mem 2 Cores 2.30GHz
Everything is working fine (yay!) 95% of the time. Now my issue is with some big queries with multiple users. I’ve set the indice configuration to be made with 280 * 5GB indices with 3 shards and 1 replica. This correspond to around 1.21Billions document.
When a user decides to make a large request the elasticsearch bursts, CPU goes to 100% and it’s not really possible to query graylog in the meantime. Is it possible to do something about that? The option I would be looking is something like either:
- killing the request after a running time of X secs and leave the DB alone
- a smart management that would reduce the priority of long running queries
Has something been done on this subject?