Pfsense snort, logs not parsing correctly

Graylog Central (peer support)
1

Hi all,

I have some problems with my graylog server 2.4.6. My pfsense machine, with snort installed, is sending its logs to my graylog server. I have created a new stream that seperates the snort logs from the original stream. So i get a nice overview of all my snort logs.

I followed this topic: Pfsense Snort logs not parsing (Resolved) for configuration, but my snort logs are not parsed correctly even in the simulator. I do have the pipeline applied to the snort stream. I can’t figure it out by myself and i’m out of ideas. Here is what i get from the simulator. I get the same result in de snort logstream. I get only 4 fields:

22
Schermafbeelding 2018-11-30 om 16.37.22.png2834×866 197 KB

And here is the code im using for the pipline:

rule "Extract Snort alert fields"
when
  has_field("message")
then
  let m = regex("\\[(\\d+):(\\d+):(\\d+)\\] (.+?) \\[Classification: (.+?)\\] \\[Priority: (\\d+)]: \\<(.+?)\\> \\{(.+?)\\} (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))? -> (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))?\\R?", to_string($message.message));

  set_field("snort_alert", true);

  set_field("generator_id", m["0"]);
  set_field("signature_id", m["1"]);
  set_field("signature_revision_id", m["2"]);

  set_field("description", m["3"]);
  set_field("classification", m["4"]);
  set_field("priority", to_long(m["5"]));
  set_field("protocol", m["7"]);

  set_field("src_addr", m["8"]);
  set_field("src_port", to_long(m["10"]));

  set_field("dst_addr", m["11"]);
  set_field("dst_port", to_long(m["13"]));
end

I did notice that i have an extra field (the snort[30224]: part) but even if i remove that part, the results are the same. I also have the correct order in System-> Configuration. So 1. Message Filter Chain -> 2. Pipeline Processor. Any help would be appreciated.

2

Did you verify your regex ?

3

Are you sending logs from Barnyard or syslog ?

4

I’m sending logs from the pfsense syslog. I didn’t verify my regex? Maybe a stupid question, but how do i do that?

5

I was using barnyard but that shouldnt matter as long as your extractors are correct.

6

At first look, I don’t see “:” after priority in your message.

\\[Priority: (\\d+)]:

But first test it with regec tester, as @hkj suggested.
Be careful, java need double escape (\\).

7

I have literally no idea how this works. I pasted the regex code in the regular expression field and the message in the string field and i get “Your regular expression does not match the subject string.” So my regular expression is not working then?

1 Like
8

Yes, you have to use a correct rexep what match with your message.
As I found, there is at least one different between your message and regex.

9

Yes i noticed the extra snort[xxxx]: part but even if i remove that part, the results are the same.
It’s the exact message as @hkj used and the exact same regex code so it should work.

10

So i copied the regex code from the original site (https://github.com/Graylog2/graylog-guide-snort) and removed the snort[xxxx]: part en it worked! I thought i tried that before but nonetheless it works now in the simulator. But my question now is: How do edit that regex code to ignore that first snort[xxxx]: part?

11

I think the better way if you understand the regex working. Of course, we can do it, but I think the goal is not that.

12

Got it working with the following code:

rule "Extract Snort alert fields"
when
  has_field("message")
then
  let m = regex("^.{13}\\s?\\[(\\d+):(\\d+):(\\d+)\\] (.+?) \\[Classification: (.+?)\\] \\[Priority: (\\d+)] \\{(.+?)\\} (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))? -> (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))?\\R?", to_string($message.message));

  set_field("snort_alert", true);

  set_field("generator_id", m["0"]);
  set_field("signature_id", m["1"]);
  set_field("signature_revision_id", m["2"]);

  set_field("description", m["3"]);
  set_field("classification", m["4"]);
  set_field("priority", to_long(m["5"]));
  set_field("protocol", m["6"]);

  set_field("src_addr", m["7"]);
  set_field("src_port", to_long(m["9"]));

  set_field("dst_addr", m["10"]);
  set_field("dst_port", to_long(m["12"]));
end

Thnx for the help!

2 Likes
13

Great :slight_smile: good to know :slight_smile:

14

@lllgeenideelll:
Heh, spotted another Dutchie :smiley:

Great job on that example code. Thanks for sharing!

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.