Background
I am trying to override the default timestamp Graylog assigns messages with the one written in the message itself.
To achieve this I read the official docs:
https://docs.graylog.org/en/3.3/pages/extractors.html#the-standard-date-converter
However I am still unable to correctly set the timestamps.
Messages
Most of messages start like the following:
2021-02-14T06:07:58.521009+00:00 myApp00_rest_of_log
There are other types of messages that don’t adhere to this standard, but that is for latter.
To extract the timestamp from the message I have created the following extractor:
The RegEx does it’s job nicely, however it’s the converter that’s killing it.
Problem
As you can see I am using the converter:
yyyy-MM-ddTHH:mm:ss.S
This doesn’t work. I have also tried the following variations:
yyyy-MM-dd'T'HH:mm:ss.S
yyyy-MM-dd HH:mm:ss.S
yyyy-MM-dd'T'HH:mm:ss.SSSSSS
yyyy-MM-dd HH:mm:ss.SSSSSS
yyyy-MM-ddTHH:mm:ss.SSSSSS
At this point you might have picked up a couple of things:
- I have 2 problems: the “T” character in the middle and the precision that comes after the seconds (has 6 numbers, instead of just 3 (\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{6}))
- I am totally lost and I am just guessing random combinations at this point
OK but flawed solution
The “OK” solution would be to use the “Flexible Date” converter, however if I do that I will lose the precision that comes after the seconds (those 6 decimals mentioned in point 1).
Unfortunately for me, I need that amount of precision in my system, so I can’t just cut out the last 6 decimals.
Questions
- How can I fix my converter so it can properly override the Timestamp with the precision I need?