Hi All. It’s come time for us to setup logging in a new build we’re doing that’s spread out across 2 Data Centers. After a lot of googling, I haven’t found a way of running a Graylog setup across 2 DCs, at least not easily anyway, due to Elastic Search not really supporting this kind of setup (Cross DC clustering). Elastic Search Cross cluster searching looked promising, but couldn’t find much info on it in relation to Graylog and unfortunately running 1 standalone Graylog setup in each Data Center isn’t an option.
In our case, we have dual 1Gbps Direct Connects into AWS at each Data Center and a single 1Gbps dedicated link between the 2 DCs. So I was looking into a hybrid type setup, where we run our Elastic Search cluster in our VPC, whilst keeping Graylog on prem. Kind of like below:
Initially a single Graylog instance in each DC would be enough for us, but it will be behind a load balancer from the start, so adding more instances should be straight forward. Graylog and MongoDB will be clustered across DCs, with a third MongoDB instance in AWS to keep an odd number.
The end goal is the have resources in each DC send their logs to the local Graylog instance, which in turn sends the data to the shared Elastic Search. A user can then log into the web interface in either data center and be able to search logs from both DCs, and we don’t have to flood the link between the Data Centers sending logs from DC2 to DC1 for example. Hopefully this setup gives us better availability and scalability by easily being able to add nodes as required.
Before I get to far down the garden path, is something like this feasible, or are there better ways to be doing it? Ideally, I’d like to keep it all on prem, but it doesn’t seem (easily) possible with Elastic Search, so putting just Elastic Search and MongoDB in AWS whilst keeping Graylog on prem seems to be the most economical way I can think to do it… if it’s possible.
Would love some feedback Thanks!