Hello Graylog Community. I have a pipeline processing a stream to drop messages containing specific source IPs. I expected that once the pipeline was kicked off, if I went into that stream I would be able to query those IPs which I removed via the pipeline processing rules and get no results. However I still get results. Sample pipeline rule:
rule "Vuln Scanner IP removal"
When
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”)
then
drop_message();
end
I expect the stream to no longer have any of these specific source IPs since it should be dropping those messages. The rule is setup as stage 0 for the pipeline.
I guess you did not really understand the function of the function has_field. This function is similar to the _exists_ query of Elasticsearch. It returns true or false if the given field exists or not. So in your case you are checking if there is a field with the name “src_ip=10.x”. You are not checking the value of that field. For your case, you should do something like this:
rule "Vuln Scanner IP removal"
when
has_field("src_ip") && (
to_string($message.src_ip) == "10.x" ||
to_string($message.src_ip) == "10.y" ||
to_string($message.src_ip) == "10.z"
)
then
drop_message();
end
Notice the brackets around the block with the IP comparisons. This allows to connect the OR operation of the IP checks with the has_field(“src_ip”) with an AND operation
