Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question. Don’t forget to select tags to help index your topic!
1. Describe your incident:
Getting Duplicate logs… we have recently set up Graylog 4.3 multinode environment having one master, master-data and data node. we are seeing duplicate logs in dashboard. we have set up stream for GELF and SYSLOG with having separate index sets and have also selected “Remove matches from ‘All messages’ stream” in stream set up… how would I stop having duplicate logs ?
Thanks for prompt reply… yes I have GELF and SYSLOG with having separate index sets… but shouldn’t “Remove matches from ‘All messages’ stream” settings does removes duplicates ???
Or is there a way I can stop having duplicates ?
It will be good to have some workaround in place because having duplicate logs on screen some times annoying users while they are running any search query and looking for some logs…
From what information is shown, I’m not sure if its a Setting/Configuration issue. What I have experienced with Dup’s message was either misconfiguration with Streams/Indices, Log shippers.
Pipelines for work-around might be you best bet, but that just a patch, its not fixing this issue.
I went through the link you provided as well before and in each discussion I found that if you have set up stream store data for separate index set then you will see duplicate data each for stream/index… (Nothing helpful solutions) so I guess I should stop using stream and its settings “Remove matches from ‘All messages’ stream” because that is not working as it suppose to…