Pfsense snort, logs not parsing correctly


(Geenidee) #1

Hi all,

I have some problems with my graylog server 2.4.6. My pfsense machine, with snort installed, is sending its logs to my graylog server. I have created a new stream that seperates the snort logs from the original stream. So i get a nice overview of all my snort logs.

I followed this topic: Pfsense Snort logs not parsing (Resolved) for configuration, but my snort logs are not parsed correctly even in the simulator. I do have the pipeline applied to the snort stream. I can’t figure it out by myself and i’m out of ideas. Here is what i get from the simulator. I get the same result in de snort logstream. I get only 4 fields:

And here is the code im using for the pipline:

rule "Extract Snort alert fields"
when
  has_field("message")
then
  let m = regex("\\[(\\d+):(\\d+):(\\d+)\\] (.+?) \\[Classification: (.+?)\\] \\[Priority: (\\d+)]: \\<(.+?)\\> \\{(.+?)\\} (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))? -> (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))?\\R?", to_string($message.message));

  set_field("snort_alert", true);

  set_field("generator_id", m["0"]);
  set_field("signature_id", m["1"]);
  set_field("signature_revision_id", m["2"]);

  set_field("description", m["3"]);
  set_field("classification", m["4"]);
  set_field("priority", to_long(m["5"]));
  set_field("protocol", m["7"]);

  set_field("src_addr", m["8"]);
  set_field("src_port", to_long(m["10"]));

  set_field("dst_addr", m["11"]);
  set_field("dst_port", to_long(m["13"]));
end

I did notice that i have an extra field (the snort[30224]: part) but even if i remove that part, the results are the same. I also have the correct order in System-> Configuration. So 1. Message Filter Chain -> 2. Pipeline Processor. Any help would be appreciated.


(Haider) #2

Did you verify your regex ?


(Haider) #3

Are you sending logs from Barnyard or syslog ?


(Geenidee) #4

I’m sending logs from the pfsense syslog. I didn’t verify my regex? Maybe a stupid question, but how do i do that?


(Haider) #5

I was using barnyard but that shouldnt matter as long as your extractors are correct.


#6

At first look, I don’t see “:” after priority in your message.

\\[Priority: (\\d+)]:

But first test it with regec tester, as @hkj suggested.
Be careful, java need double escape (\\).


(Geenidee) #7

I have literally no idea how this works. I pasted the regex code in the regular expression field and the message in the string field and i get “Your regular expression does not match the subject string.” So my regular expression is not working then?


#8

Yes, you have to use a correct rexep what match with your message.
As I found, there is at least one different between your message and regex.


(Geenidee) #9

Yes i noticed the extra snort[xxxx]: part but even if i remove that part, the results are the same.
It’s the exact message as @hkj used and the exact same regex code so it should work.


(Geenidee) #10

So i copied the regex code from the original site (https://github.com/Graylog2/graylog-guide-snort) and removed the snort[xxxx]: part en it worked! I thought i tried that before but nonetheless it works now in the simulator. But my question now is: How do edit that regex code to ignore that first snort[xxxx]: part?


#11

I think the better way if you understand the regex working. Of course, we can do it, but I think the goal is not that.


(Geenidee) #12

Got it working with the following code:

rule "Extract Snort alert fields"
when
  has_field("message")
then
  let m = regex("^.{13}\\s?\\[(\\d+):(\\d+):(\\d+)\\] (.+?) \\[Classification: (.+?)\\] \\[Priority: (\\d+)] \\{(.+?)\\} (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))? -> (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})(:(\\d{1,5}))?\\R?", to_string($message.message));

  set_field("snort_alert", true);

  set_field("generator_id", m["0"]);
  set_field("signature_id", m["1"]);
  set_field("signature_revision_id", m["2"]);

  set_field("description", m["3"]);
  set_field("classification", m["4"]);
  set_field("priority", to_long(m["5"]));
  set_field("protocol", m["6"]);

  set_field("src_addr", m["7"]);
  set_field("src_port", to_long(m["9"]));

  set_field("dst_addr", m["10"]);
  set_field("dst_port", to_long(m["12"]));
end

Thnx for the help!


(Haider) #13

Great :slight_smile: good to know :slight_smile:


(Tess) #14

@lllgeenideelll:
Heh, spotted another Dutchie :smiley:

Great job on that example code. Thanks for sharing!


(system) closed #15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.