Flexibly Parse Date

Graylog Central (peer support)
1

We have a number of Palo Alto’s forwarding their logs to Graylog. Unfortunately, the timestamps are in EST and Graylog interprets that as UTC. as a result, the log messages appear 4 hour prior. Our network team is not willing to change the timezone on the Palo Altos so I need to have another solution.

I created an extractor using Copy Input on the timestamp field and have tried both the date convertor and the flexibly parse date convertor to update set the timezone to EST. however, neither of these seemed to work and the timestamp still appears in UTC

Using the current version of Graylog (v2.2.3) 3 node cluster with a 3 node ES cluster in the backend.

Any help would be greatly appreciated.

2

The parse_date() function and the Date converter have parameters for specifying the timezone of the parsed date.

You can use those to put the correct date into the timestamp field of your messages.

3

Hi jochen
So what you are suggesting is using the Pipeline functions to modify the timezone ?

and thanks for the quick reply

4

Either use the message processing pipelines or create a Copy Input extractor with a Date converter which converts the existing date.

5

I have tried the Copy Input extractor with the Date convertor and that didnt appear to work.
Example Message: 2017-06-14T09:56:44.000Z
Extractor Type: copy input
source field: timestamp
store as field: timestamp
Extraction strategy: cut
Convert to date type:
format string: yyyy-MM-dd HH:mm:ss.SSS
Time Zone: Toronto

I would assume that should work, unless I am pulling out of the wrong field.

6

timestamp is a non-string field (it contains a Date object), so extractors won’t run on that.

Either use processing pipeline rules and the parse_date() function or try extracting the timestamp from your message or full_message fields as a string (using a Regex extractor and a Date converter).

1 Like
7

and thats why it didnt work.

Thank you. I extracted the timestamp and then used the Date Convertor on the new field

its working now. thank you for you help

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.