Input Extractor and Pipeline Rule

[jrussell (Justin Russell)]

1. Describe your incident:

I have a single pipeline with the following extractor:

Trying to extract data from records_properties into , leaving the original intact.

I’m trying to write a write a pipeline rule that will drop messages where properties_smbCommandMajor == 9. properties_smbCommandMajor is a derived field from the extractor. I can search for properties_smbCommandMajor:9 and see results, but I cannot figure out the right syntax for the rule. Here’s what I think should work, but is not:

rule "drop if properties_smbCommandMajor == 9"
when
    $message.properties_smbCommandMinor == 9
then
    drop_message();
end

2. Describe your environment:

• OS Information:
  Private Build 17.0.3 on Linux 5.15.0-1014-azure
• Package Version:
  Graylog 4.3.3+86369d3
• Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?
I’ve tried as many variations of $message.properties_smbCommandMinor == 9 that I can think of.

4. How can the community help?
Please help me write the rule to drop messages where properties_smbCommandMajor == 9.

[tmacgbay]

I wonder if it is a type issue where the data for properties_smbCommandMinor is stored as keyword… if so you could try putting quotes around the number…

$message.properties_smbCommandMinor == "9"

[jrussell (Justin Russell)]

Thank you for the quick reply. I tried $message.properties_smbCommandMinor == "9" as well as to_string($message.properties_smbCommandMinor) == "9". I also tried without $message like "properties_smbCommandMinor" == "9" and "properties_smbCommandMinor" == 9

[tmacgbay]

Check your message processor configuration, you may want “Message Filter Chain” before “Pipeline Processor”

you can find out what is happening in a pipeline rule by using the debug() function… so something like this:

rule "check properties_smbCommandMajor"
when

    has_field("properties_smbCommandMinor")

then
 // use $ tail -f /var/log/graylog-server/server.log to watch for the results of the below debug message
 //
 debug(concat("============ properties_smbCommandMinor: ",to_string($message.properties_smbCommandMinor)));

end

[jrussell (Justin Russell)]

Thank you again for your help. I tried your suggestion, but the condition has_field("properties_smbCommandMinor") didn’t result in anything being appended to the log. When I used the following code:

rule "check properties_smbCommandMajor"
when
    $message.properties_smbCommandMinor == 9
then
 // use $ tail -f /var/log/graylog-server/server.log to watch for the results of the below debug message
 //
 debug(concat("============ properties_smbCommandMinor: ",to_string($message.properties_smbCommandMinor)));
end

The following is appended to the log:
WARN [EqualityExpression] left expression evaluated to null, returning false: $message.properties_smbCommandMinor

Does this mean the field $message.properties_smbCommandMinor does not exist? Could it be that pipelines rules are evaluated before extractors, in which case the field doesn’t exist yet?

I’ve attached a screenshot of the extractor configuration and you can see "smbCommandMajor":5 in the example message.

[tmacgbay]

Check to make sure that Message Filter Chain comes before Pipeline Processor - under System/Configurations…

[jrussell (Justin Russell)]

Thanks so much for your help, tmacgbay. It was the message processors configuration. Once I adjusted the order, the following rule worked as expected:

rule "drop message when properties_smbCommandMajor == 9"
when
    to_string($message.properties_smbCommandMajor) == "9"
then
    drop_message();
end