pfSense SNORT SYSLOG Log Parsing

Graylog Central (peer support)
1

Hi All :slight_smile:
I am very new to Graylog, very quickly realizing how powerful the product it

I have a pfSense Firewall with SNORT installed, which works great. SNORT outputs the logs to the System Logs in pfSense and pfSense outputs the logs via SYSLOG to Greylog

All of the logs are arriving correctly within Graylog, however for the life of me I cannot parse the correct fields so I can start to get some useful data. A few examples of the MESSAGE field within the log are

snort[87537]: [136:1:1] (spp_reputation) packets blacklisted [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 2.3.4.5:16856 -> 1.2.3.4:2222

snort[87537]: [119:4:1] (http_inspect) BARE BYTE UNICODE ENCODING [Classification: Not Suspicious Traffic] [Priority: 3] {TCP} 4.5.6.7:63651 -> 1.2.3.4:8080

I am using a Pipeline with the following rule:

rule “Extract Snort alert fields”
when
has_field(“message”)
then
let m = regex("^\s?\[(\d+):(\d+):(\d+)\] (.+?) \[Classification: (.+?)\] \[Priority: (\d+)] \{(.+?)\} (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:(\d{1,5}))? -> (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:(\d{1,5}))?\R?", to_string($message.message));

set_field(“snort_alert”, true);

set_field(“generator_id”, m[“0”]);
set_field(“signature_id”, m[“1”]);
set_field(“signature_revision_id”, m[“2”]);

set_field(“description”, m[“3”]);
set_field(“classification”, m[“4”]);
set_field(“priority”, to_long(m[“5”]));
set_field(“protocol”, m[“6”]);

set_field(“src_addr”, m[“7”]);
set_field(“src_port”, to_long(m[“9”]));

set_field(“dst_addr”, m[“10”]);
set_field(“dst_port”, to_long(m[“12”]));
end

Any ideas on what I need to do to get this working ?

Cheers, Scott :slight_smile:

2

Hiya Scott,

I think your regex is a little too complicated - you also anchored it on something that’ll never match because \s? just means “is there a space”…

For example, taking these messages:

snort[87537]: [136:1:1] (spp_reputation) packets blacklisted [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 2.3.4.5:16856 -> 1.2.3.4:2222

snort[87537]: [119:4:1] (http_inspect) BARE BYTE UNICODE ENCODING [Classification: Not Suspicious Traffic] [Priority: 3] {TCP} 4.5.6.7:63651 -> 1.2.3.4:8080

You can immediately see they have things in common - it’s also better to use grok because it lets you use some pre-built patterns, that are just much easier to read. For example:

grok("^snort[%{INT}]: [%{INT:generator_id}:%{INT:signature_id}:%{INT:signature_revision_id}] (%{WORD:module}) %{DATA:description} [Classification: %{DATA:classification}] [Priority: %{INT:priority}] {%{WORD:protocol}} %{IP:src_addr}:%{POSINT:src_port} -> %{IP:dst_addr}:%{POSINT:dst_port}", to_string($message.message), true)

This returns a map that you can use with set_fields to set all the named captures all at once. You may have to escape some things in the GROK pattern (such as the {} around the protocol) but I think this will get you what you’re looking for.

Alternatively you can store the entire pattern by using the “Grok Patterns” page, save it as something like “SNORTLOG” and then you can just grok("%{SNORTLOG}", ...) to make it a bit easier to alter the pattern if need be.

3

That’s exactly what I was thinking: this smells like it needs a nice grok-pattern.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.