We collect logs on site and send them to a local Graylog instance and do the parsing and normalisation there. We then send the logs to a clustered Graylog environment in Azure to perform analysis and alerting on the logs. We are trying to load balance as then we can split the log traffic equally between two Graylog nodes and get increased performance cheaper.
We have set up a traffic manager in Azure which has a DNS name, we then send the logs from the local Graylog to the DNS name of the load balancer via GELF TCP + TLS, this means that a connection is opened to one server but is never dropped and the traffic only ever goes to one node until the connection is manually stopped.
Does anyone load balance with GELF TCP + TLS? Or can anyone provide me with any other ways of splitting my traffic evenly between two remote nodes.
Why do you think that load-balancing TLS over TCP is different from load-balancing TCP?
In the end, it makes sense for any TLS client to keep connections alive, because connection initialization is relatively expensive when using TCP and even more when using TCP+TLS.
If you don’t want to profit from that optimization, you’ll have to use clients which close the connection after each message (or after a certain number of messages) or a load-balancer which does TLS-termination and is aware of the application protocol being used, so that it can decide which packets to send to which backend service.
I’m sorry for not being completely clear. What I don’t fully understand is how load balancing TCP works, even after reading the information in the link provided by jan. I just added + TLS as in my situation we are using encryption.
As TCP is a connection orientated protocol, how does Graylog or the load balancer ensure that not just one connection is created and used. We have tried Azure traffic managers but they use DNS to route the traffic and as TCP created a connection to the first IP behind DNS and the connection was never terminated so the traffic is not balanced.
Would HA proxy terminate the connections every so often to ensure traffic is balanced?
