Beats messages are duplicated in multi-server setup

I have constructed a lab identical to the configuration show in Architectural Considerations. I’m running the following:

  • Graylog 3.0.1 on three (3) servers
  • elasticsearch 6.7.1-1 on three (3) servers
  • HAProxy 1.8.19

All systems are running CentOS 7.6.1810.

I have systems running journal beat 6.7.1-1 which are forwarding messages to the HAProxy server. My HAProxy server config has a listen block for the beats as follows:

listen beats
    bind *:5044
    mode tcp
    #option tcpka
    #option tcplog
    #balance source
    server graylog01 maxconn 100 check
    server graylog02 maxconn 100 check
    server graylog03 maxconn 100 check
    timeout connect 20s
    #timeout check 20s
    timeout client 86400s
    timeout server 86400s

I’m getting duplicated messages on most but not all message transmissions. I’ve tried using a lot of different configurations and options to try and address this to no avail.

I’ve tried:
server graylog01 maxconn 100 check
server graylog02 maxconn 100 check backup
server graylog03 maxconn 100 check backup

as well as, simply commenting out graylog02 and graylog03 to force all messages through a single node, but the problem persists.

The messages are not complete duplicates from what i can see, but the only field that is different is journalbeat_read_timestamp.

I’ve removed all my streams and just have the default stream configured. There are no dashboards, no pipelines, etc. I was looking to sort out message(s) reception first before moving on to other configuration options.

Any help is much appreciated.


I may have figured it out - it appears that the Elastic JournalBeats is the culprit in sending duplicated messages. :frowning:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.