Creating Stream Rules Using Pipeline Fields

I created a field using pipeline to get the duration data from the message content.

rule "rule_duration"
  has_field("message") AND to_long($message.level) == 6
  let result = split(" ",to_string($message.message))[9];

I want to create a stream to store the messages their duration value is greater than 0. When I configure the stream by pipeline condition, the messages I send are not redirected to the stream.

Can not I use a field, that is created by pipeline, in stream rules? If yes can you recommend any suggestion to me?


What’s the order of message processors in your Graylog cluster?
You can check that on the System/Configurations page in the web interface.

Also, please provide some example messages.

Processor order is like this:

  1. Message Filter Chain
  2. Pipeline Processor

When I change the order of them, the fields that created by pipeline are not exist in any stream. In both cases I cannot make the stream work.

With that order, the Stream filter (the component running the stream rules and assigning streams to a message) are running before the pipeline rules.
If you want to use one or more fields created by a pipeline rule in your stream rules, you have to swap the order.

Alternatively, you can assign streams in your pipeline rules:

1 Like

I tried the solution you suggested and it was successful.

Thanks a lot

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.