I have a Graylog installation that can handle well over 25.000 msg/sec.
I am now experimenting with setting up a dedicated Logstash cluster. So something like this:
Filebeat (n hosts) -> Logstash Cluster -> Graylog
The biggest issue now is that I can not send more than 2.500 msg/sec from one host to Graylog. No matter how big the host is (4, 8, 16, … CPUs), no matter how small the messages are, there is always this 2.500 msg/seg limit.
I’ve even created messages with only 1 Byte payload, the limit here is 2.800 msg/sec.
I can send messages from, for example, 4 different hosts. Resulting in 10.000 msg/sec (4 x 2.500). Graylog can handle this easily.
So my question is: Why can one host send no more than 2.500 msg/sec via GELF?
