Adding timezone to FortiGate logs

[dansaj]

Hello,
I try to configure pipeline that it adds timezone to FortiGate logs which looks like below:

date=2019-05-15 time=10:56:59 devname=“…” devid=“…” logid=“…” type=“traffic” subtype=“local” level=“notice” vd=“root” eventtime=1557910619 srcip=… srcname=“…” srcport=137 srcintf=“lan” srcintfrole=“lan” dstip=… dstport=137 dstintf=unknown-0 dstintfrole=“undefined” sessionid=14191988 proto=17 action=“deny” policyid=0 policytype=“local-in-policy” service=“NetBIOS” dstcountry=“Reserved” srccountry=“Reserved” trandisp=“noop” app=“netbios forward” duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat=“unscanned” devtype=“Windows PC” devcategory=“Windows Device” osname=“Windows” osversion=“7” unauthuser=“…” unauthusersource=“kerberos” mastersrcmac=“…” srcmac=“…” srcserver=0

I searched forum and finally tried rule that looks like below:

rule “fortigate timestamp”
when
true
then
let build_message_0 = concat(to_string($message.date), " ");
let build_message_1 = concat(build_message_0, to_string($message.time));
let new_timestamp = parse_date(value:to_string(build_message_1), pattern:“yyyy-MM-dd HH:mm:sss”, timezone:“Europe/Warsaw”);
set_field(“timestamp”, new_timestamp);
end

But when I simulate it I got error:

gl2_processing_error
For rule ‘fortigate timestamp’: In call to function ‘parse_date’ at 7:20 an exception was thrown: Invalid format: " "

Any ideas what should I change?

Regards
Daniel

[jan (Jan Doberstein)]

did you before use a key-value parser to create the fields from message?

[dansaj]

To be honest… no.
I’m newbie in this…

[jan (Jan Doberstein)]

so you do not have the data available that you are working with - I guess.

[dansaj]

i have data. I have all logs.
But, what is wrong in my rule?

[jan (Jan Doberstein)]

Sorry that I wasn’t verbose enough.

My meaning of data is that it looks for me like you have the content of the field message separated into different fields. That you have those fields you refer too available.

Means you have a field date with the value 2019-05-15 and a field time with the value 10:56:59 and so on for all key=value data you send in the message field. Because such is not done automatically you need to tell Graylog that it should do this.

Does it make more sense now?

[dansaj]

I don’t know if I understand you correctly…
I copied content of the field message from Search Tab in Graylog and tried to work on it. Is this wrong behaviour?

[jan (Jan Doberstein)]

Did you have multiple fields visible when you look at you messages like in the screenshot below (that is just an example not how winlogbeat data might look like).

Or did you have only 4 fields visible?

[dansaj]

I have many fields visible when I’m looking at messages. Exactly like in your screenshot.

[jan (Jan Doberstein)]

how does your field date and time look like?

[dansaj]

Ok.
My fields “date” and “time” looks like in the attached screenshots.

[BlueTeamNinja (aBe)]

I don’t know if you solved your problem or not - but my Stage 0 for Fortinet devices (almost all have the weird timestamp):

rule "fortigate timestamp"
when
    (has_field("devname") && has_field("date") && has_field("time")) or
    (has_field("device_id") && contains(to_string($message.device_id), "FNTORONTO1", true) && has_field("date") && has_field("time")) or
    (has_field("device_id") && contains(to_string($message.device_id), "FNTORONTO2", true) && has_field("date") && has_field("time"))
then
    let build_message_0 = concat(to_string($message.date), " ");
    let build_message_1 = concat(build_message_0, to_string($message.time));
    let new_timestamp = parse_date(value:to_string(build_message_1), pattern:"yyyy-MM-dd HH:mm:sss", timezone:"America/Toronto");
    set_field("timestamp", new_timestamp);
end

I add one for the devices in each timezone as they don’t typically change in/out too often.

EDIT: I was also advised to use the rename function in the past, I just haven’t revisited this in awhile and I know the above works in our environment.

[dansaj]

Thank you for help. It works