Currently in our production we are running single node Graylog + ES + MongoDB. We just want to move beyond as our log flow increases day by day.
Our idea is to bring another node with Graylog + ES + MongoDB. Now, for both ES, Graylog we are getting more performance and space to store more logs. There is no problem in forming ES cluster. But Graylog clustering is not happening.
I configured Primary node as is_master=true and other with is_master=false.
I followed following article.
But the second Graylog node is not joining to the cluster and If I login to the Graylog WebUI I am only seeing only one node. Please note that this is to expand the capacity not to provide the HA.
Here we are not using and LB and want to use one WebUI to manage both the ES nodes. My question here, though master graylog manages both ES, will it share the Graylog processing loads (We have a lot of GROK)
did both Graylog Nodes share the same MongoDB? I want to highly this part of the reference:
As nearly all configuration of your Graylog setup is stored in MongoDB, make sure it is reachable for every new Graylog node you add to the setup. If you will be adding just one Graylog node, you can move to the next step. If you are adding more than one node or want high-availability (HA) for MongoDB, please take a look at our FAQ on scaling MongoDB.
Nope. Both have its own MongoDB. MongoDB replicas I am not going to enable. Only Graylog and ES will form the cluster. All the metadata of Graylogs will be stored in its own MongoDB. Do you think this will create a problem as these DB’s are not talking each other ?
of course this will be a problem. If both Graylog-instances are configured differently, inconsistencies will happen, causing your system to not work like intended. As far as my experience with Graylog goes, Graylog definelty does not like it when it’s nodes go out of sync, because e.g. the MongoDB cluster fails and configuration changes are not applied on all nodes because of that. So, I would recommend you to set up a MongoDB replica, because else your system might not work (at least I never heard of two independent Graylog-instances writing into the same indexes, but a Graylog-Dev will probably be able give a more exact answer than mine)
What you could do is to let each Graylog-instance write into it’s own index-set, but then you won’t be able to search all data at once, since Graylog would then just act as if it were two independent installations.
@derPhlipsi already said the important part @xorloader41 - if both Graylog instances did not share the MongoDB they are both not a cluster but single node installations.
