Graylog Data Retention and Compress

Graylog Central (peer support)
8

When you grow:

  • put streams with the same set of field on the same index-set. Winlogbeat to winlogbeat, squid on their own, firewall on it’s own and so on.
  • Separate your Elastic/Openseach from your Graylog Nodes
  • put a small loadbalancer in front of your Graylog

Quick sketch, how I do tune Openseach/Elastic next

  • set the index set for daily rotation if suitable. For two GB of Data you might increase that to a week.
  • for each 20-30GB of data per rotation on a index set there should be one shard
  • for each 20 shards one GB of heap on the OS/Elastic.
  • 50% of RAM for the java-heap, rest for filesystemcaches via OS on OS/Elastic.
1 Like