I am looking to set up Graylog, a Bigger Production Setup on Kubernetes. I didn’t find any official setup documentation for Kubernetes on the Graylog website. Let me know if we have any.
What I mean by a Bigger Production Setup is: Graylog: 1 Master, 1 web interface, and 10 Data nodes Mongo: 1-3 Mongo nodes Elasticsearch: 3 Master and 5 ES data nodes.
Is it recommended to set up a bigger Graylog cluster on Kubernetes?
You can use this documentation here for Docker and/or Kubernetes.
A brief overview in order.
1.Setup ES cluster first
2.Setup MongoDb replica set
3.Setup Graylog cluster
There is not a multi-node cluster setup guide for Docker/Kubernetes that I know of. You might want to mention this here detail what you would like.
What I would do is get an idea how to setup a multi-node cluster and adapt it to Docker/Kubernetes environment, that way you know all the graylog environment variables. Just an idea.
Also, Is it recommended to set up a bigger Graylog cluster on Kubernetes? Asking because document related to Graylog setup on kubernetes is very rare on Google.
I would first look into how many devices are going to sending logs, get a average of logs per day ( 5 Gb , 50GB, 1 TB, etc…) and build out from there. What I have done was start with the basic setup 3 ES, 3 GL/Mongo nodes. Insure you have enough resources ( CPU, RAM, HDD). Try not to slam Graylog cluster right away with logs. This will help adjusting resources so the cluster doesn’t fall on its face.
Is you want to set up a Graylog from scratch I’d recommend not to go for Elastic, but for Opensearch. I is quite clear, that this will be the future database for logs. Migrating sometimes is quite hard.