Apply Correct timezone and convert date

Given this string:
Mar 28 2020 18:52:43

I am able to extract and make a date field using this grok extractor in a pipline line:
let fptimestamp = grok(pattern:"%{FP_DATETIMESTAMP:FP_TimeStamp:date;MMM dd yyyy HH:mm:ss}" , value: message_field, only_named_captures: true);

This returns:
FP_TimeStamp: 2020-03-29T01:52:43Z
The Actual timestamp on this record is:
2020-03-28 11:52:47 -07:00

In the original message the timestamp is in UTC but no timezone is specified. So in the grok statement , i am guessing it is converting it to UTC again. How do I add the correct timezone -7.00, so that the grok statement will convert the time to the correct time as per my timezone? Or do I need to add another line somewhere to do this?

Well after beating my head against my desk for a few hours I figured out a makeshift solution. Instead of using a pipeline rule, I created a grok extractor on the input to just extract “Mar 28 2020 18:52:43” from the message with no conversion. I then created a second extractor to copy the field created by the first extractor. On the second extractor I used a date converter. Catch was I had to set the timezone to “Argentina” and the locale to “English”. My copied timestamp now appears as:

2020-03-28 18:52:43 -07:00

100% correct and offset is correct as well.

After this I created a pipeline rule to remove the original field that this data was copied from. This works because my extractors are processed before my pipeline.

Would love it if someone could help me do this correctly, but am posting it as possible workaround. Hopefully it helps someone. Reading though numerous other posts of people have similar issues, it appears that the parse_date functions in Java can be quite fussy and buggy.

he @williamk

depending how your initial timestamp source look like you might be able to make your move in a single processing pipeline rule.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.