Using Extractors with grok Pattern

[Alexnf (Alex S)]

Hi all,

i am quite new to this and would need some help understanding the Grok Patterns. I have tho following:

date=2020-06-30 time=09:21:14 devname="600E" devid="FG6H0E5819904479" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1593501674337461223 tz="+0200"

i want to create an Extractor using Grok Pattern.

I started like this (I want year,month,day in separated fields):

%{YEAR:IncYear}[-]%{MONTHNUM:IncMonth}[-]%{MONTHDAY:IncDay}

which worked fine. How do I proceed from there i tried many of the time patten but none seem to work, or I (most likely use them wrong).

Maybe a kind soul can give me a push by highlighting how i can format the first 3-4 fields, I would highly appreciate it .

[shoothub]

Don’t try to extract by one one, best way is to create extractor with key-value. This way, it extract all fields with key=value to separate fields.

https://docs.graylog.org/en/3.3/pages/extractors.html?highlight=key%20value#automatically-extract-all-key-value-pairs

By the way, fortigate uses 2 separate field to setup date and time, you can use this pipeline rule to concat them, and use it as timestamp:

rule "fortigate_timestamp"
when
  has_field("devname") AND has_field("date") AND has_field("time")
then
    let build_message_0 = concat(to_string($message.date), " ");
    let build_message_1 = concat(build_message_0, to_string($message.time));
    let new_time = parse_date(value: build_message_1, pattern:"yyyy-MM-dd HH:mm:ss", timezone:"Europe/Bratislava");
    set_field("timestamp", new_time);
end

Don’t forget to check processing order, so pipeline is after extractors in list
https://docs.graylog.org/en/3.3/pages/pipelines/stream_connections.html?highlight=processing%20order#the-importance-of-message-processor-ordering

[Alexnf (Alex S)]

I can’t thank you enough. Works like a charm and is exactly what I needed.