Could you post the ASA_Pipeline rule and a typical message it should process? The rest that you describe seems right to me unless you have a typo. Before even posting you can change the condition between the when and then to always be true (pretty sure you can just put in the word true and nothing else.) just to see if that might be blocking…
The reason this is such a simple (and seemingly non-sensical) rule is:
Inputs auto-start when you restart the system. I have some inputs that I need to run as-needed because they create too much data. If I send those messages to the Drop_Message rule by default, I can save my database from getting accidentally filled up after a system restart.
This is an easy way to get my feet wet and figure out how to do pipelines.
I’m stumped. You said you see messages coming into the stream - double check the pipeline connection - go into the pipeline, clicked on “Edit connections” and make sure your stream is listed. Here is an example of “Linux Events” pipeline connected to “Linux Stream” Stream
Actually I think your right.
After some more reading I see that the order works best when fields in a pipeline are in play. Then realizing @danmassa7 is wanting to just drop all message so the order of the message processors would not help much. Perhaps the stage of the pipeline might be something to look at?
However, if you prefer to use the original stream matching functionality (i.e. stream rules), you can configure the Pipeline Processor to run after the Message Filter Chain (in the Message Processors Configuration section of the System → Configurations page) and connect pipelines to existing streams. This gives you fine-grained control over the extraction, conversion, and enrichment process.
I use Streams to direct my messages to particular index (a crucial feature). And I also want to use pipelines because that’s a really cool feature in Graylog. Why would anyone ever want to run the Pipeline Process before the Message Filter Chain? But I guess that’s the default order out-of-the-box. Whatever the reason, the default order out-of-the-box is completely dysfunctional if you want to use the pipeline feature.
Thank you everyone for all your help! I now have the Pipeline Processor after the Message Filter Chain and things are working well.