About MPS performanced and it related hardware requirements

(Charles Deng) #1

I know different hardwares, different graylog configurations and different messages make difference. but is there any performance data about input MPS(messages per second) achieved for GELF over HTTPs and it related server hardwares can be referenced ?

(Jochen) #2

No, there are no public benchmarks about that, but feel free to share yours so that other users can check them out!

(Charles Deng) #3

Currently, I am using 3-MongoDB KVM cluster+3-Elasticsearch KVM Cluster+3-Graylog KVM Cluster.
each MongoDB with 2 cores, 4GB RAM, ES with 8 cores, 64 GB RAM, Graylog with 64 Cores, 64GB RAM. 3 primary shards with 1 replication for each primary shard.

I was rong and i must have drunken the input MPS required was 6K MPS instead of 100 MPS, Currently my throughput problem is not on server side, but agent side. i firstly start 3 agent processes to input with GELF over HTTPs by shell program with curl & awk, each curl message about 1.2KBytes. it only achieve 10+ MPS. when i start 90 agent processes to input, I achieved almost 100 + MPS. but it seems it is not fessible to using shell scripts to achieve 7.2K MPS which three hosts to input. and i don’t if i should turn to UDP.

(system) closed #4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.