FortiGate firewalls send syslog data without a timezone field. The date and time are correct, but since there is no timezone listed Graylog assumes it is UTC, and it converts the timestamp.
Here is an example of the message data the FortiGate sends:
Here is the timestamp after Graylog has ingested it:
2020-06-06 11:12:01 -05:00
I’ve created a pipeline with the following rule:
rule "fix_timestamp" when has_field("timestamp") then let incorrect_timestamp = to_string($message.timestamp); let correct_date = substring(incorrect_timestamp, 0,11); let correct_time = to_string($message.time); let concat_date_time = concat(correct_date,correct_time); let corrected_timestamp = to_date(concat_date_time, "yyyy-MM-dd HH:mm:ss", "America/Chicago"); set_field("timestamp", corrected_timestamp); end
I’m not getting any errors, but logs in the stream don’t seem to be updating.
What am I doing wrong?
Is there a better way to do this?
After looking at the raw syslog input using tshark, the FortiGate is sending timezone data, so I’m even more confused now.
date=2020-06-06 time=17:53:41 tz="-0500"
I edited my pipeline rule to this, just to see if I could get it to save the data to a new field, but I’m still not getting anything.
rule "fix_timestamp" when true then set_field("timestamp_custom", parse_date(concat(concat(to_string($message.date)," "),to_string($message.time)), "yyyy-MM-dd hh:mm:ss", "America/Chicago")); end
Even trying a rule that doesn’t do anything other than set a custom field, isn’t working. New logs coming in the stream are not showing up with the “pipetest” field I’m trying to generate.
rule "fix_timestamp" when true then set_field("pipetest","rule processed"); end
But when I run the pipeline simulation it does show the rule is processing correctly.
Simulation results These are the results of processing the loaded message. Processing took 672 µs. Changes in original message 1b6e78e0-a856-11ea-bf02-b8ac6f83164c Added fields pipetest rule processed