The ‘Pipeline Processor’ has to run after the ‘Message Filter Chain’ to be able to access fields created by extractors. This then forces a change in the OOTB processing order found in the ‘System / Configurations’.
Any particular reasons why this OOTB setting was chosen instead of the other order ?
By doing this order change configuration, am I losing/braking something else ?
Yes, this enables you to access fields created in pipeline rules in your stream rules.
You won’t be able to access any message fields in extractors or stream rules which have been created by pipeline rules.
So if I get this right:
The only reason I decided to look into changing the OOTB “Message Processors Configuration” order, was because of the following scenarios:
While using a ‘Graylog Collector Sidecar’ for Windows and successfully dropping messages using a rule pipeline for winlogbeat messages based of winlogbeat_level == “Information” (we don’t want to hold on Windows info level in Graylog), I then decided to do the same for filebeat messages (from ad hoc Windows apps) based on extractors field adhoc_apps_level == “info”. This last pipeline rule was not dropping messages (like the first one did), UNTIL I reverse the order as explain in #2. Then, both scenarios worked.
How can I achieve the same scenarios (drop winlogbeat & filebeat ‘info’ messages) with keeping the default order as explain in #1.
Migrate your extractors to pipeline rules which run in a stage before the rules in which you drop the respective messages.
Thanks jochen, I have followed you recommendations and it’s working perfectly!
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
