Replace string in rule

[MyCodeRocks]

I know its Xmas (Merry Xmas everyone) but this post is a testament to how frustrated I am. I am sure this is a dumb question but I have spent two days on this.

Use Case:
I have a regex that I use to get some values out a Snort log. It works 99% but for one field. There is a destination port: dst_port. If I use regex101.com this value is extracted and is shown as Group 9 (if you count as an array starting at 0). I am doing a normal:

set_field("dst_port", m["9"]);
9 = the grouping of the result. All other results work per their grouping numbers.

Example:

set_field("src_port", m["5"]); works

I then thought maybe for some reason this datatype is different and tried:

set_field("dst_port", to_long(m["9"])); - in case its a datatype formatting issue for whatever reason and that doesn’t help.

It then lead me to a “quick fix” I also store the port number with the " : " as I use that for something else so example:
:8080 or :8180
I added a second stage to the pipeline (the rule explained above is 0 Stage) that had the following code using a replace:

rule "remove :"
when
has_field("message")
then
let dst_port = regex_replace(":", to_string($message.message), "");
end

Now I don’t know if I am understanding the documentation incorrectly for both how you run rules in stages as well as the syntax for the regex_replace function. But it does not remove the " : " from the :<port_number> – what I am trying to do is transform :8080 to just 8080 and there I have a usable port number.

The rule is pretty simple - set / let the dst_port be the outcome of the regex above. The regex above is simple find " : " and replace it with " " (nothing) to remove the " : " at the beginning of the string.

Now:

1. The normal set_field("dst_port", m["9"]); should be working off the bat… so I was trying the second rule (and learning at the same time)
2. The second rule with regex_replace doesn’t seem to do anything at all. I know I am using $message.message which is the entire message not just that section of the message. This was desperation as well as I wasn’t sure if I did $message.long_dst_port (this stores the :8080) if it would reference or look at that field that was set in rule 1.

Would really appreciate the help or pointers. This thing is driving me crazy and I am sure its something so stupid that I am missing.

[jan (Jan Doberstein)]

if you want to remove something from the field dst_port you need to put that in the regex_replace rule.

Even when the tests do not use the naming of the variables ( https://github.com/Graylog2/graylog2-server/blob/master/graylog2-server/src/test/resources/org/graylog/plugins/pipelineprocessor/functions/regexReplace.txt ) I would write pattern, string and replacement into the rule.

http://docs.graylog.org/en/3.1/pages/pipelines/functions.html#regex-replace

Personal I would check if I can replace the : with a specific character first. “nothing” might be handled different and maybe just replcae that with a woudl show you if the logic is working at all.

You did not share your first rule at all so it is not possible to help with that.

[MyCodeRocks]

Thank you very much @jan I finally to work what I did -

First:
After I set the field for the original data :8080
I then create variable with that value:
let desp = m(4); ——- m(4) being the group in the regex.
This gave me a variable to work with.

Then at the end of setting fields I then create another variable:
let dest_fix regex.replace = (, m(4), “”)

My regex was also wrong so fixed that…

Then set the dst_port to the fixed value 8080.
Set_field dst_port = dest_fix.

This works I get a clean 8080 in the value.

Just a question for my understanding in setting a variable in the regex,replace when you use $message., after you set a field cam you reference it in regex.replace with $message.<field_name> or does one have to first set a variable for it like I did above and reference that.

Just want to follow the best method.

[jan (Jan Doberstein)]

after you set a field cam you reference it in regex.replace with $message.<field_name> or does one have to first set a variable for it like I did above and reference that.

both is possible. No good or bad is given here. Take what you can better read.