Need help regarding tuning Graylog Cluster

Graylog Central (peer support)
1

Hi,

Here we have already done with the configuration part but for proper functioning of it need some guidance. Because I don’t want to face the same problem that I was faced in previous graylog version.

  • Please suggest the ElasticSearch tuning that we need to update in our ES_Cluster, we are using default
    conf file with cluster nodes IPs only.

  • we are assuming 10k -15k messages per second, so please suggest what we need to tune in server.conf &
    other conf if required.

  • Data Retention - for retaining 6 months data, is it ok to have 1 master with 2 data nodes ?
    Please suggest what do you recommend ?

Our Current Configuration (We are having big Infrastructure and configure the things accordingly)

  • Graylog Web-Interface / node-1 & node-2 -> Having 16 GB RAM and 8 Core CPU
    graylog version - 2.4.5

  • Elasticsearch version - 5.6.4
    ES Master - Having 16 GB RAM and 4 Core CPU
    ES Data Node(2) - Each having 32 GB RAM and 8 Core CPU

  • Mongo version - 3.2 (having a single node with 8 GB RAM and 2 Core CPU

Thanks

2

Not sure if you can have that period of time with just two node. You might have around 217 TB or more data with your ingest rate. The sharding of Elasticsearch will give you some guidance

3

Thanks @jan we will think to implement this in our infrastructure.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.