We’re currently doing some load testing to find out how Graylog reacts to an overload situation.
Setup: graylog sidecar with filebeat sends logs to graylog with kafka journal enabled and elastic search.
Once we send messages faster than Elasticsearch can injest the kafka journal fills up and and starts to discard messages.
- can we have a Graylog setup where Graylog slows down injestion from clients and doesn’t discard messages? Can this be achieved by disabling the Kafka journal?
- I am missing a chapter describing how Graylog reacts to high load. I found out about discarding messages only by reading the configuration file.
- I’d like to know if Graylog implements for example back-pressure strategies (when using file-beat this sounds like a reasonable alternative on the client side to discarding messages on the Graylog server side)