Redundant messages in Default Stream despite "Remove matches from ‘Default Stream’" being checked

[marziglt (Leonard Marzigliano)]

Using Graylog 6.1, we’ve configured the message routing by sorting five different log types into five streams/index-sets. After learning that Stream Rules will become a deprecated feature, we instead accomplished this by creating a single Pipeline connected to all five streams and added five rules to Stage 0 to route them accordingly.

Each of the streams we created has the option checked for “Remove matches from ‘Default Stream’ (Don’t assign messages that match this stream to the ‘Default Stream’.” - yet still the messages are sent to the Default Stream as well as the routed stream, creating redundancy.

Is this because we skipped out on using the soon to be deprecated Stream Rules? Can we somehow keep the Pipeline Rule routing but eliminate the redundancy caused by the failure to remove matches from the Default Stream?

We tried adding a separate Pipeline/Rule that drops the redundant messages from the Default Stream but it instead dropped all specified messages from both streams.

[patrickmann]

route_to_stream pipeline function has an optional parameter remove_from_default. You need to use that instead of the checkbox in the stream, if you are relying on the pipeline to perform the routing.