# Pipeline Rule - Drop messages with field containing IP

**URL:** <https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065>\
**Category:** Graylog Central (peer support)\
**Tags:** pipeline-rules, debuggingpl\
**Created:** [March 10, 2021, 2:46pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065 "2021-03-10T14:46:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![markus2066](https://avatars.discourse-cdn.com/v4/letter/m/ba9def/32.png) [@markus2066](https://community.graylog.org/u/markus2066)\
**Post date:** [March 10, 2021, 2:46pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/1 "2021-03-10T14:46:46Z")

</div>

Hi All,

I have been trying for a number of hours without success to implement a pipeline rule which drops messages when a field contains an IP address from a specific subnet. I have found a few similar examples in posts but they don’t seem to be having the desired effect.

The graylog input is a sonicwall firewall sending logs via udp syslog. Various extractors are used to parse the traffic, I am trying to drop messages when a field “Int\_Src\_IP” equals an address in the 172.16.40.0/24 subnet.

Message Filter Chain is before Pipeline Processor in the Message Processors Configuration section.

The two different rules (and some variations) I have tried to use are:

rule “Drop NAT from specific Subnet”  
when  
contains(to\_string($message.Int\_Src\_IP), “172.16.40.\*”, false)  
then  
drop\_message();  
end

And:

rule “Drop NAT from specific Subnet”  
when  
has\_field(“Int\_Src\_IP”) AND $message.Int\_Src\_IP == “172.16.40.\*”  
then  
drop\_message();  
end

As additional background re my config, I also have the following pipeline rule in place (working) which drops any messages which dont contain have “NAT Mapping” in the msg field.

rule “Drop all but Rule”  
when  
NOT contains(to\_string($message.msg), “NAT Mapping”, false)  
then  
drop\_message();  
end

I have tried these rules in both the same and separate stages but am still seeing the unwanted messages with 172.16.40.0/24 addresses being stored.

I would be very grateful is someone could point me in the direction of where i might be going wrong.

Regards

Mark

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [March 10, 2021, 11:28pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/2 "2021-03-10T23:28:55Z")

</div>

First thought is remove the \* - I don’t think you can use that in a quoted area

`contains(to_string($message.Int_Src_IP),"172.16.40", false)`

You can use `contains()` or perhaps `starts_with()` would be more efficient.

---

<div class="post-metadata">

**Author:** ![markus2066](https://avatars.discourse-cdn.com/v4/letter/m/ba9def/32.png) [@markus2066](https://community.graylog.org/u/markus2066)\
**Post date:** [March 11, 2021, 11:20am UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/3 "2021-03-11T11:20:30Z")

</div>

Hi Tmacgbay,

Thanks for your suggestion but neither seem to work.

I’m having some doubts about Graylog picking up the variable i’m using. Is there any documentation which shows how to debug or validate a variable?

Graylog is currently being spammed with log entries from a particular set of devices and due to run out of space before the end of the day so really desperate to get this resolved by filtering out these messages.

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [March 11, 2021, 11:43am UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/4 "2021-03-11T11:43:09Z")

</div>

you can use the `debug()` function and watch what it’s output is in the Graylog logs:

`tail -f /var/log/graylog-server/server.log`

```
rule "Drop NAT from specific Subnet"
when
    starts_with(to_string($message.Int_Src_IP), "172.16.40")
then
    debug("Source IP match:");
    debug(to_string($message.Int_Src_IP));
    drop_message();
end

```

if nothing shows you can change the when to `has_field("Int_Src_IP")` just to make sure you get to the debug

[Debug Function - Graylog 4.0](https://docs.graylog.org/en/4.0/pages/pipelines/functions.html#debug)

---

<div class="post-metadata">

**Author:** ![markus2066](https://avatars.discourse-cdn.com/v4/letter/m/ba9def/32.png) [@markus2066](https://community.graylog.org/u/markus2066)\
**Post date:** [March 11, 2021, 6:03pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/5 "2021-03-11T18:03:36Z")

</div>

Hi Tmacgbay,

Thanks for clarify the debug process, so i have tried with both the starts\_with and has\_field but my server.log file is almost strangely quite. I don’t believe I have previously made any changes to internal logging config, do I need to enable debugging at all? The last entry’s in this log are from the server starting following a reboot today 5 hours ago.

My log4j2.xml file contains(in case this is relevant):

\<?xml version="1.0" encoding="UTF-8"?\> 

```
    <!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. -->
    <Memory name="graylog-internal-logs" bufferSize="500"/>
</Appenders>
<Loggers>
    <!-- Application Loggers -->
    <Logger name="org.graylog2" level="info"/>
    <Logger name="com.github.joschi.jadconfig" level="warn"/>
    <!-- This emits a harmless warning for ActiveDirectory every time which we can't work around :( -->
    <Logger name="org.apache.directory.api.ldap.model.message.BindRequestImpl" level="error"/>
    <!-- Prevent DEBUG message about Lucene Expressions not found. -->
    <Logger name="org.elasticsearch.script" level="warn"/>
    <!-- Disable messages from the version check -->
    <Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/>
    <!-- Silence chatty natty -->
    <Logger name="com.joestelmach.natty.Parser" level="warn"/>
    <!-- Silence Kafka log chatter -->
    <Logger name="kafka.log.Log" level="warn"/>
    <Logger name="kafka.log.OffsetIndex" level="warn"/>
    <!-- Silence useless session validation messages -->
    <Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/>
    <Root level="warn">
        <AppenderRef ref="rolling-file"/>
        <AppenderRef ref="graylog-internal-logs"/>
    </Root>
</Loggers>

```

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [March 11, 2021, 6:32pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/6 "2021-03-11T18:32:56Z")

</div>

You shouldn’t need to turn anything on. did you try changing the when part to make sure you can look at what is coming in? Be sure to comment out the `drop_message()` since it may catch more than what you want to drop. Or create/apply a new temp rule:

```
rule "Drop NAT from specific Subnet-TEST"
when
    has_field("Int_Src_IP")
then
     let debug_message = concat("Source IP match-test: ", to_string($message.Int_Src_IP));
     debug(debug_message);

end
```

---

<div class="post-metadata">

**Author:** ![markus2066](https://avatars.discourse-cdn.com/v4/letter/m/ba9def/32.png) [@markus2066](https://community.graylog.org/u/markus2066)\
**Post date:** [March 12, 2021, 12:13pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/7 "2021-03-12T12:13:54Z")

</div>

Hi Tmacgbay,

It works! Thank you so much for your help I couldn’t have done it without it.

So I was using the following debug rule in stage 1, but was still getting no output in the server.log file:

rule “Debug Field”  
when  
has\_field(“Int\_Src\_IP”)  
then  
let debug\_message = concat(“Debug-Out:”, to\_string($message.Int\_Src\_IP));  
debug(debug\_message);  
end

But had this rule in stage 0:

rule “Drop all but Rule”  
when  
NOT contains(to\_string($message.msg), “NAT Mapping”, false)  
&&  
NOT contains(to\_string($message.msg), “Assigned IP”, false)  
&&  
NOT contains(to\_string($message.msg), “DHCP”, false)  
then  
drop\_message();  
end

The issue (which I still don’t really understand) seemed to be the order. As a test I applied the debug rule (using a different/relevant field) to a different stream which had no other pipeline rules configured, this resulted in the expected debug output being generated in server.log.

Returning to the pipeline rule I was having issues with I removed the stage 0 rule and placed it in stage 1, then applied the debug rule at stage 0. This then generated the debug output in server.log.

I was then able to replace the debug rule at stage 0 with the following which is working at desired:

rule “Drop NAT from Listed Subnets”  
when  
contains(to\_string($message.Int\_Src\_IP), “172.16.40”, false)  
OR  
contains(to\_string($message.Int\_Src\_IP), “172.16.50”, false)  
OR  
contains(to\_string($message.Int\_Src\_IP), “172.16.60”, false)  
then  
drop\_message();  
end

Again thank you so much.

Mark

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [March 26, 2021, 12:13pm UTC](https://community.graylog.org/t/pipeline-rule-drop-messages-with-field-containing-ip/19065/8 "2021-03-26T12:13:56Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.