Hello forum users,
i’m trying to setup a pipeline, which is connected to the “All Messages” Stream.
I’m trying to write a pipeline rule that copies a message into two streams:
In Stage 1, the message will be stored in a shortterm stream.
Stage 2 should then check whether the message contains an IP or not. If the message has an IP, this and other personal data should be anonymized, if not, the message should simply be copied into a long-term stream.
My rules are the following:
Stage 1
rule “shortterm log”
when
$message.source == “server.example.com”
then
route_to_stream(“shortterm”);
end
Stage 2
rule “anonymize-ip”
when
has_field(“ip”) && $message.source == “server.example.com”
then
let ip_address = to_ip($message.ip);
set_field(“ip”, ip_address.anonymized);
set_field(“ip_city_name”, “”);
set_field(“ip_country_code”, “”);
set_field(“ip_geolocation”, “”);
route_to_stream(“long term log”);
end
rule "longterm-logs"
when
!(has_field("ip")) && $message.source == "server.example.com"
then
route_to_stream("long term log");
end
Are my rules correct or is there a better way to copy one message to two streams?