Best way to avoid bursts


Here’s roughly my setup:

  • 3 * graylog web 4GB 2 Cores 2.90GHz
  • 3 * elasticsearch 16GB Mem 2 Cores 2.30GHz

Everything is working fine (yay!) 95% of the time. Now my issue is with some big queries with multiple users. I’ve set the indice configuration to be made with 280 * 5GB indices with 3 shards and 1 replica. This correspond to around 1.21Billions document.

When a user decides to make a large request the elasticsearch bursts, CPU goes to 100% and it’s not really possible to query graylog in the meantime. Is it possible to do something about that? The option I would be looking is something like either:

  • killing the request after a running time of X secs and leave the DB alone
  • a smart management that would reduce the priority of long running queries

Has something been done on this subject?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.