Testing Pipeline Effect on Stream

Graylog Central (peer support)
1

Hello Graylog Community. I have a pipeline processing a stream to drop messages containing specific source IPs. I expected that once the pipeline was kicked off, if I went into that stream I would be able to query those IPs which I removed via the pipeline processing rules and get no results. However I still get results. Sample pipeline rule:

rule "Vuln Scanner IP removal"
When
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”) ||
has_field(“src_ip=10.x”)
then
drop_message();
end

I expect the stream to no longer have any of these specific source IPs since it should be dropping those messages. The rule is setup as stage 0 for the pipeline.

Any recommendations greatly appreciated.

2

Hey @Nick.Ray.NTX,

I guess you did not really understand the function of the function has_field. This function is similar to the _exists_ query of Elasticsearch. It returns true or false if the given field exists or not. So in your case you are checking if there is a field with the name “src_ip=10.x”. You are not checking the value of that field. For your case, you should do something like this:

rule "Vuln Scanner IP removal"
when
  has_field("src_ip") && (
  to_string($message.src_ip) == "10.x" ||
  to_string($message.src_ip) == "10.y" ||
  to_string($message.src_ip) == "10.z"
  )
then
  drop_message();
end

Notice the brackets around the block with the IP comparisons. This allows to connect the OR operation of the IP checks with the has_field(“src_ip”) with an AND operation :slight_smile:

Greetings - Phil

3

Thats the one! Thanks