there’s an option in the graylog script graylog-ctl which comes with the OVA to set the time zone. If you’re not using the OVA one must set the time zone inside the config files but that will require looking at the docs.
you can extract the date first as a string (a new field), then use a copy extractor with date converter. Note that the new field must be a string; the date converter does not seem to work from date type to date type. The date converter allows you to specify the time zone.
But actually log still parsed in UTC if timezone not specified. Is there a way to specify default timezone? Other than parse as string -> convert string to date?
Why config’s root_timezone do not impact extractor default timezone?
I never thought of doing any kind of date/time conversion on-the-fly at the extractor level; I just let it ride as it wants and then Graylog allows me to search using my time zone ranges (PST in my case) and I get correct hits instead of doing UTC searches. Screenshot… hope I understood the issue correctly.
Your log message do not have time stamp. So graylog set own timestamp when get this message. In my case log message which come to graylog has own time stamp which set by application (nginx error log) and i want that it will be the same in graylog so i use message extractor on field message:
2017/12/30 15:19:57 [error] 27739#0: *3722503 limiting requests, excess: 0.062 by zone “wp-login”, client: 10.189.252.6
And in message time in CST (-5) but it is not specified in log and when it parsed by Graylog he has no idea which timezone use and consider it as log in UTC format, but it actually CST.
The best decision here is change log format in application that add timezone after time, but it impossible as nginx do not permit change format of error log. So i need anyway tell Graylog that my time not in UTC timezone, default timezone in graylog config file is not impact this, graylog-ctl i no have as this is not OVA installation.
Seems format-date can help, i have create pipeline rule with one “Stage 0” attached to “All messages” stream.
I need convert timestamp field but only for those messages which with wrong timezone (UTC) it can be sorted by field “type” = “nginx_error” is there possible setup rule to find value in fields?
I only found has_field function will see only whether my field “type” exist. Is there a way filter “type” fields only which has “nginx_error” value?
For check timezone in date in timestamp field i should look into field value, but which function can do this? I found function “has_field” but it is check only if field present. Filed “timestamp” present for all messages, i need sort those who has format “dd/MMM/yyyy HH:mm:ss” and not change other like “dd/MMM/yyyy HH:mm:ss Z”
I think the easiest way would be making a stream with condition source is equal to the source id, and use the pipeline rule for that. Most log sources will send logs with reasonable timestamps, so this would not be too difficult.
I also thought about separate stream, but maybe it possible sort logs rely on field value in pipeline only? At all is it possible that pipeline look into field value? Maybe some function? Like has_field or match_field_value?
