Beats messages are duplicated in multi-server setup

Graylog Central (peer support)
1

I have constructed a lab identical to the configuration show in Architectural Considerations. I’m running the following:

  • Graylog 3.0.1 on three (3) servers
  • elasticsearch 6.7.1-1 on three (3) servers
  • HAProxy 1.8.19

All systems are running CentOS 7.6.1810.

I have systems running journal beat 6.7.1-1 which are forwarding messages to the HAProxy server. My HAProxy server config has a listen block for the beats as follows:

listen beats
    bind *:5044
    mode tcp
    #option tcpka
    #option tcplog
    #balance source
    server graylog01 192.168.4.34:5044 maxconn 100 check
    server graylog02 192.168.4.35:5044 maxconn 100 check
    server graylog03 192.168.4.36:5044 maxconn 100 check
    timeout connect 20s
    #timeout check 20s
    timeout client 86400s
    timeout server 86400s

I’m getting duplicated messages on most but not all message transmissions. I’ve tried using a lot of different configurations and options to try and address this to no avail.

I’ve tried:
server graylog01 192.168.4.34:5044 maxconn 100 check
server graylog02 192.168.4.35:5044 maxconn 100 check backup
server graylog03 192.168.4.36:5044 maxconn 100 check backup

as well as, simply commenting out graylog02 and graylog03 to force all messages through a single node, but the problem persists.

The messages are not complete duplicates from what i can see, but the only field that is different is journalbeat_read_timestamp.

I’ve removed all my streams and just have the default stream configured. There are no dashboards, no pipelines, etc. I was looking to sort out message(s) reception first before moving on to other configuration options.

Any help is much appreciated.

Thanks

2

I may have figured it out - it appears that the Elastic JournalBeats is the culprit in sending duplicated messages. :frowning:

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.