Strange problem with pipeline

Templates and Rules Exchange Pipeline Rules
1

Hi.

My Graylog server (5.2.4) routes logs from sidecar (audibeat) to two streams - streams A and B (by streams rules) and its ok. Stream A should keep oryginal log but stream B should keep only few fields. So I created rules for pipeline like this:

rule “ssh alerts cut”
when
contains(
value: to_string($message.“auditbeat_auditd_data_terminal”),
search: “ssh”,
ignore_case: true
)
AND
contains(
value: to_string($message.“auditbeat_auditd_result”),
search: “fail”,
ignore_case: true
)
then
remove_field(
field : “^(?!auditbeat_event_original$).*$”,
invert : false
);
end

This rules are attached to pipeline to stream B and cut log as I wish… but it turns out that stream A is also cut up and the data in it is exactly the same as in stream B. Why? My message processors order are: 1.Stream Rule Processors 2. Message filter chain 3. Pipeline Processors. Any help please?

2

The same message can be mapped to multiple streams. Routing to a stream does not create a copy. Use the clone_message function to create a copy.

3

Thx for your explanation. Now I’ve done it a bit roundaboutly using ‘create a new message’ in pipeline.