I have a question about what order the rules execute in a particular stage.
Can you conceptually view all rules within a stage as happening in parallel? Or are they guaranteed to run sequentially in the order listed?
If they run in the listed sequence, should there be a feature enhancement request to be able to change the order the rules are listed in the UI? Or am I missing something, because the only way to change the order is to start dumping rules out and then adding them back in.
I believe they run in parallel. What would otherwise be the purpose of “stages”?
However, I have no idea how graylog handles conflicts between these rules.
I guess the fastest way is to run some experiments with the Pipeline simulator ?
Heads up: Pipeline simulator may fail if your rules modify timestamp - details here.
Although it is described in documentation, it still wasn’t clear to me (always take me longer!) so I did some testing on this.
Rules in a stage are generally run in parallel - or at least you should assume so. If you want Rule_A to run before Rule_B you need to make sure that Rule_B is in a higher numeric stage number (stages can be numerically negative).
If you shunt the message to another pipeline via a command in it’s current pipeline, the message will finish all commands and stages in the current pipeline, and THEN it will start the next pipeline you send it to.
HOWEVER. If you have a message that starts traversing two or more pipelines at the same time, stages run in lockstep/parallel.
So a message would execute all rules in Pipeline_D stage 1 and Pipeline_E stage 1 before it would execute any rules in any subsequent stages… and so on. This means that if Pipeline_E did not have a stage two and pipeline_D did (and they both had a stage 3) then the rules in pipleine_D stage 2 would run and then both pipelines would run stage 3 in parallel. Oy, that is a lot of words…