Hello,
I am struggling in understanding the reason for the order of the message processor configuration.
The sequence in the standard configuration is as follows:
# Processor Status
1 AWS Instance Name Lookup active
2 GeoIP Resolver active
3 Pipeline Processor active
4 Message Filter Chain active
I have now built a stream rule that decides which stream the message should be routed to based on the tags field.
Then I created a pipeline that was connected to this stream with a test stage:
rule "test"
when
true
then
set_field("test", "test");
end
After some time I found out that this doesn’t work, because by default the pipeline runs before the Message Filter Chain, which e.g. decides depending on rules to pack messages into streams. As soon as I changed the order under System -> Configuration it works.
But I am not yet happy with this solution. I want to understand why it is set up this way. There must be one or more reasons.
How should I handle it? Change it? Or do the routing in the pipelines? What are the advantages and disadvantages?
Some matching links and information:
https://docs.graylog.org/en/3.2/pages/pipelines/stream_connections.html#the-importance-of-message-processor-ordering