Pipeline rule: Unable to route messages to another stream

Graylog Central (peer support)
1

Hello,
I’m trying to route some messages to a separate stream using a pipeline rule. Do you have any idea why this doesn’t work?

rule "Route to Extra Stream"
when
    regex("Folders_\\d+", regex_replace("([^:\\s]+)\\s*:[^|{\\n}]+[|{\\n}]?", to_string($message), "$1", true)).matches == true
then
    route_to_stream(id: "68762c0e60e3431b6931defc");
end

The messages are initially routed to a stream using stream rules, and that is the stream to which the pipeline is connected.
The messages are not routed to the stream 68762c0e60e3431b6931defc

If I add a set_field(“test”,“ok”) after route_to_stream and use the rule simulation, I see “test” being added to the message.

Any ideas?

Thanks,

Al

  • OS Information: Ubuntu 22.04
  • Package Version: 6.3.1
  • Message Processors: Message Filter Chain, Stream Rule Processor, Pipeline Processor
2

I can’t see anything wrong with the rule; and the test field confirms that the when clause is successfully evaluated.

Does the user have write access to the stream?

3

Not sure if that is what you mean, but my user has admin rights.

I added a set_field like this:

rule "Route to Extra 1"
when
    regex("Folders_\\d+", regex_replace("([^:\\s]+)\\s*:[^|{\\n}]+[|{\\n}]?", to_string($message), "$1", true)).matches == true
then
    route_to_stream(id: "68762c0e60e3431b6931defc");
    set_field("pipeline_executed","Route to Extra 1"
end

I see messages comming to the original stream that should trigger the rule. They do not have the “pipeline_executed” field. If I copy the same message to rule simulation, the simulation adds the field. It’s like the condition is met only in the simulator.

I have other pipelines routing messages to other streams working fine.

4

When rules runnin simulator but not in production its usually one of two things. either the message processing order is incorrect. Or the data is somehow different when its going through processing than after its stored (and you are using these stored values in the testing). I would use the debug pipeline function to write some of the field values to server.log and see if they look the same when passing through the pipeline.

1 Like