We have defined two pipelines, both on stream ‘All Messages’, with different stages (0 and 1)
Picked up by Pipeline (stage 0)
&& contains(to_string($message.source),“MyDummyHostName”) == true
route_to_stream(id:“5aa0ec7f3893e1097cc20f65”, remove_from_default:true); //security
results in Message:
And the above message is as well picked up by different pipeline (stage 1)
pipeline (stage 1)
&& has_field(“level”)== true
&& contains(to_string($message.source),“MyDummyHostName”) == false
route_to_stream(id:“5aa0e05f3893e1097cc1a988”, remove_from_default:true); //syslog_default_new
My understanding is that the pipeline ‘security’, running as stage 0, connected to ‘All Messages’ stream, is picking up the message at first, doing all the stuff defined within the rule. Once that pipeline is finished, any other pipeline connected to the same stream running on the same stage are now run.
In my case, there is no pipeline anymore running on stage 0.
Now, the second pipeline ‘syslog’, running on stage 1 runs. Somehow that pipeline is working on the same message already handled by pipeline ‘security’, which had as well moved that message to a different stream (and deleted from original stream!).
IMHO that’s a fault, this approach results in having a message with wrong defined fields, as ‘hostname’ is now set to ‘syslog’.
So, did I found a bug or is it ‘works as designed’.
Am I’m doing something wrong?
Sure it’s possible to extend the ‘when’-definitions to not run into that issue, but finally that would increase overhead and maintenance work as those statements are getting more complex.