Hello and good morning from Rostock Germany.
I am searching for a possibility to prevent duplicates via pipeline in Graylog 3.
I only found information how to clean up, when it already happened - I would like to avoid writing the data set to the database at all.
My record has a unique id and the probability is given that the record will be sent twice.
So I would like to investigate via pipeline something like
when $message_unique_id is_in_stream(id:“123456789”)
Is there something like that?