Pipeline rule does not apply to stream messages

[morel666]

My log messages look like:

f77ecb70 09:19:09 ASCleanr: Scheduled event in: 300 sec(s)

so I have created pipeline rule:

rule "Extract message fields"
when
    true
then
    let app_log_pattern = "%{NOTSPACE:sessionId} %{TIME} %{NOTSPACE:module}\\s*\:\\s*%{GREEDYDATA:message_text}";
    set_fields(grok(app_log_pattern, to_string($message.message)));
end

I have created stream, that filters all messages from this particular application and pipeline, that is connected to this stream, with only one rule assigned on Stage 0, and no other stages.
When I test this rule with Simulator - then everything is ok, message is properly splited according to grok pattern, and log from simulation trace confirms that everything should work properly:

0 μs
Starting message processing
51 μs
Message fe1f7a61-c490-11e7-bdba-00505680993b running [Pipeline 'app1 - pipeline' (5a02c8af8321f9d50e9d29ae)] for streams [5a02c7198321f9d50e9d27f2]
128 μs
Enter Stage 0
141 μs
Evaluate Rule 'Extract message fields' (5a02d01a8321f9d50e9d31bc) in Pipeline 'app1 - pipeline' (5a02c8af8321f9d50e9d29ae)
161 μs
Evaluation satisfied Rule 'Extract message fields' (5a02d01a8321f9d50e9d31bc) in Pipeline 'app1 - pipeline' (5a02c8af8321f9d50e9d29ae)
174 μs
Execute Rule 'Extract  message fields' (5a02d01a8321f9d50e9d31bc) in Pipeline 'app1 - pipeline' (5a02c8af8321f9d50e9d29ae)
609 μs
Completed Stage 0 for Pipeline 'app1 - pipeline' (5a02c8af8321f9d50e9d29ae), continuing to next stage
616 μs
Exit Stage 0
622 μs
Finished message processing

But if I view messages on Stream, then it looks as this rule not worked. How to debug that and find what is wrong?

[morel666]

The answer is: proper order of message processors. The proper configuration is: Pipeline Processor after the Message Filter Chain

[jan (Jan Doberstein)]

That highly depends on your setup. Just to have it said, not every setup need that order.

[morel666]

Could you please write a bit more about this? Just little example when Pipeline Processor should be before Message Filter, and when Message Filter should be before Pipeline Processor. I will continue to read documentation according to this info.