Pipeline rule to grok notice.log from BRO-IDS

Graylog Central (peer support)
,
1

Hi All,

I want to parse the notice log from BRO-IDS with a pipeline rule with grok. I have the following rule:

rule “Bro notice log fields”
when
has_field(“file”) &&
contains(value: to_string($message.file), search: “notice.log”, ignore_case: true)
then
let parsed_fields = grok(“(?:%{BASE10NUM:ts})\t%{NOTSPACE:uid}\t%{NOTSPACE:id_orig_h}\t%{NOTSPACE:id_orig_p}\t%{NOTSPACE:id_resp_h}\t%{NOTSPACE:id_resp_p}\t%{NOTSPACE:fuid}\t%{NOTSPACE:file_mime_type}\t%{NOTSPACE:file_desc}\t%{NOTSPACE:proto}\t(?\S+::(\S+?)+)\t(?(.*?))\t%{WORD:sub}\t%{IPV4:srcip}\t%{IPV4:dstip}\t%{NOTSPACE:dstport}\t%{NOTSPACE:count}\t%{NOTSPACE:peer_descr}\t(?\S+::(\S+?)+)\t(?:%{BASE10NUM:supress_for})\t%{WORD:Dropped}\t%{NOTSPACE:Country_code}\t%{NOTSPACE:Region}\t%{NOTSPACE:City}\t%{NOTSPACE:latitude}\t%{NOTSPACE:longitude}”, $message.message);
set_fields(parsed_fields);
end

I get the error: Invalid Expression (line: 6, column: 411)

image
image936×216 12.5 KB

I need some help to fix this, the grok pattern works when I put it on https://grokdebug.herokuapp.com/ with this line of log

1554394503.663211 - - - - - - - - - Scan::Port_Scan 10.3.1.19 scanned at least 15 unique ports of host 10.3.1.97 in 0m1s local 10.3.1.19 10.3.1.97 - - - Notice::ACTION_LOG 3600.000000 F - - - - -

2

The grokdebugger on heroku is not the same GROK engine Graylog uses…

In Graylog 3.0 you could use the build in Grok Debugger …

I guess it is because you use $message.message and not to_string($message.message) - but not sure with that.

3

any try to escape \.
eg. \\t instead of \t