Replace Action with pipeline / extractor

Graylog Central (peer support)
1

Hi,

I have a firewall log:
USG3 kernel: [LAN_IN-2002-R]IN=eth1.11 OUT=eth1.10 MAC=74:83:c2:d9:1d:74:02:42:c0:a8:0b:33:08:00:45:00:00:28 SRC=192.168.11.51 DST=192.168.10.101 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=36555 DF PROTO=TCP SPT=8084 DPT=53034 WINDOW=0 RES=0x00 ACK RST URGP=0

I want some information in a seperate field so I created some extractors with grok and regex.

In [LAN_IN-2002-R] the “R” is for Rejected, a “A” for Allow and “D” for Drop. So I want to replace the R for Rejected in this case. How is it possible? I think with a rule in a pipeline.

I tested this:
rule “Replace Action”
when
has_field(“message”)
then
let result = regex("^.-(.+)\].$", to_string($message.message));
set_field(“var3”, result);
end

That created a field var3 with:
{"0":"R"}

How is it possible to just set “R” or bether Rejected?

If I change has_field(“message”) to has_field(“ACTION”) the set_field does not work. ACTION is a self created extractor. It this case it is easier to replace R with Rejected instead of with the hole message.

Is it better to use extractors, pipelines/rules or a mix of both?

2

he @TWART016

I would run key-value extraction on this messages first. This way you get most parts in separate fields already.

Check what is left on the way. You might want to use regex replace for the word matching.

3

Hi @jan

what do you mean with key-value? In the extrators or pipeline?

4

he @TWART016

I would run a key-value function in the processing pipeline and in general use the the processing pipeline for such a work. It is not that easy as the extractors but more powerful.

5

Hi @jan

now I have a rule:

rule "key-value"

when
 has_field("message")
then
set_fields
        (
        fields: key_value
                (
                value: to_string($message.message),
                trim_key_chars: "\"",
                trim_value_chars: "\""
                )
        );
    remove_field(field: "TTL");
    remove_field(field: "ID");
    remove_field(field: "PREC");
    remove_field(field: "DPT");
    remove_field(field: "TOS");
    remove_field(field: "LEN");
    remove_field(field: "facility");
    remove_field(field: "level");
    remove_field(field: "RES");
    remove_field(field: "URGP");
    remove_field(field: "WINDOW");

    rename_field(old_field: "PROTO", new_field: "PROTOKOLL");
    rename_field(old_field: "SPT", new_field: "SRC-PORT");
    rename_field(old_field: "DST", new_field: "DST-IP");
    rename_field(old_field: "SRC", new_field: "SRC-IP");
   
end

This separates some values:

DST-IP
224.0.0.251

PROTOKOLL
UDP

SRC-IP
192.168.100.104

SRC-PORT
5353

message
USG3 kernel: [WAN_LOCAL-default-D]IN=eth0 OUT= MAC= SRC=192.168.100.104 DST=224.0.0.251 LEN=414 TOS=0x00 PREC=0x00 TTL=255 ID=41839 DF PROTO=UDP SPT=5353 DPT=5353 LEN=394

source
USG3

timestamp
2020-05-13 23:46:00 +02:00
  1. How can I extract the rule name: WAN_LOCAL-default
  2. The “D” behind the rule. I can’t figure out how to use regex replace to extract it. I want to replace
  • D -> Deny
  • A -> Allow
  • R -> Reject
  1. Is it possible to sort the fields?
  2. I remove some fields. How can I whitelist that instead of blacklist?
6

he @TWART016

  1. I guess regex is your friend for that
  2. use regex_replace for that
  3. the order of fields does not matter at all.
  4. it is not possible to have a rule that delete all fields that are not on the “wanted” list. This would be a feature request! https://github.com/Graylog2/graylog2-server/issues
7

Hi @jan

  1. With that regex I normally get the write string: ^.USG3 kernel: [(.+)-. This is inside my rule: let result = regex("^.*USG3 kernel: \\[(.+)-.*",to_string($message.message));
    set_field(“Rule”,to_string(result));
    The output is: {0=WAN_LOCAL-default}
    How can I remove curly brackets and the “0=”?
    Same for the field Action.

  2. Can I do if the first one is right

8

Try this.

The following matches WAN_LOCAL-default
(?<=\[).*(?=\-)

The following matches the D but would also match A or R
(?<=\-)[DAR](?=\])

9

With that two rows I get no result: {}

let rule1 = regex("(?<=\\[).*(?=\\-)",to_string($message.message));
let action = regex("(?<=\\-)[DAR](?=\\])",to_string($message.message));

I escapted the two “\” in each line.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.