Hi,
Actually when I send a message with filebeat into elasticsearch, i can define a fingerprint that allow to not duplicate the message if i send it several time.
I would like to send the same message with rabbitmq and use graylog to consume it.
But each time the message is get from the input, graylog add gl2_message_id that is different for the same message
Is there a way to not duplicate the same message into graylog ?
thanks
I think that i must use pipeline rule to drop message if the fingerprint value already exist.
But i’he no idea how to do it
Wouldn’t it be more efficient for Rabbitmq to deduplicate?
In fact the question is:
Does graylog allow to force de _id to avoid duplicate messages ?
With logstash i can force the document_id with a fingerprint function to be sure that i don’t duplicate a message
With graylog is there a way to do a similar operation ?
The aim is to be able to send the same message without to be duplicated into ES.
such is not possible with Graylog. At the time of writing this.
you might want to open a feature request for that:
I found lot of issue on this question.
I have replied on an existing issue : https://github.com/Graylog2/graylog2-server/issues/466
Hope my suggestion will help
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
