Load Balancer, Graylog and INPUTS

(Mihail Politaev) #1

Hello community. We have one Graylog node so far. But now want to scale up to 2 and put load balancer before them. One node has 1 input GELF on 12201 UDP port, how we should send logs after load balancer has been placed before 2 graylog nodes? Just send logs to load balancer, which in turn send them across all nodes behind?

And one question more. How we can check us is we actually need a load balancer and our nodes under load? Maybe some specific graylog metrics are? Because CPU not so much loaded according to Zabbix metrics.

(Jochen) #2

In theory yes, but be aware that GELF UDP is inherently unfriendly for load balancers due to its chunking characteristics. All chunks (UDP packets) of a GELF message have to be sent to the same Graylog node for processing.

The buffer utilizations (input, processing, and output buffers) are usually a good indicator.

(Matt) #3

Sorry to hijack Micha but I’m going down a similar path.

Jochen - in your experience is it advisable to send UDP traffic direct to one of the nodes? TCP traffic can be sent to the LB with no issue?

Thanks, Matt

(Jochen) #4

There should at least be a consistent distribution of packets from the same source address to the backend servers, so that all UDP packets from the same source will be sent to the same Graylog node.

I don’t want to generalize that, but at least for GELF TCP that’s not a problem.

(Mihail Politaev) #5

All chunks (UDP packets) of a GELF message have to be sent to the same Graylog node for processing.

Ok, jochen for what then we need LB before Graylog nodes? I thought in case if we have so much incoming messages but now know incoming message should be sent directly to Graylpg nodes. In which case we need LB, when too much http request? Too much users? We have 3-4 active users. We don’t need LB think and vertical scaling prefer for us?

(Jochen) #6

I don’t know any of your use cases or requirements.

That’s not what I’ve written. Additionally, there are quite a few other transport mechanisms than GELF UDP which do not have the same limitations.

(Mihail Politaev) #7

When running multiple Graylog servers a common deployment scenario is to route the message traffic through an IP load balancer.

This from Graylog documentation.

Sorry, yes. You didn’t say that messages should be send directly to graylog nodes, but say that may have problems in case if gelf messages willn’t send to the same grsylog node through lb. What you can propose? Send messages to lb and hope that all will be sent to the same graylog nodes?

(Jochen) #8

Yes. In case of GELF UDP and only all chunks of the same message, not all messages.

I can only repeat myself:

(system) closed #9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.