Best practice - Loadbalancer

[KPS]

Hi!

I did successfully setup my first production Graylog cluster (3x Graylog, 3x ES).
Everything is working fine, but I did not find a good solution to

• distribute the load
• get good availability

→ I need a loadbalancer

First try: haproxy → not suited for UDP
Second try: nginx → can loadbalance UDP, but does not preserve the source. → RAW-UDP-inputs are showing nginx-IP as source

How do you distribute the load to your cluster-nodes?

Thank you for your help!
KPS

[gsmith]

Hello,

I have seen most community members us Nginx.

Nginx Config Examples

For those of you who use Nginx as a reverse proxy or load balancer, you may find these configuration files useful: Here’s the example main nginx config I use for load balancing tcp/udp connections:

nginx.conf

user nginx; worker_processes 4; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $r…

As for load balancers, We have set Nginx in front of Graylog Web UI for that connection. As for Data/Metadata redundancy this would be configured within Elasticsearch or MongoDb.

I have used my enterprise firewall as a load balancer this would depended on how you configured you environment.

hope that helps

[KPS]

Hi!

I just found a solution to preserve the source IP:

NGINX:

  upstream graylog_syslog11524 {
    server graylog01:11524 max_fails=3 fail_timeout=30s;
    server graylog02:11524 max_fails=3 fail_timeout=30s;
    server graylog03:11524 max_fails=3 fail_timeout=30s;
  }

  server {
    listen 11524 udp;
    proxy_bind $remote_addr transparent;
    proxy_pass graylog_syslog11524;
    proxy_timeout 1s;
    error_log /var/log/nginx/graylog_syslog_udp11524.log;
  }

[gsmith]

Nice Thanks for posting your resolution to this issue. If you could make it as resolved for future search that would be great.

[KPS]

Thank you, solution-mark is set.