At this point you might have picked up a couple of things:
I have 2 problems: the “T” character in the middle and the precision that comes after the seconds (has 6 numbers, instead of just 3 (\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{6}))
I am totally lost and I am just guessing random combinations at this point
OK but flawed solution
The “OK” solution would be to use the “Flexible Date” converter, however if I do that I will lose the precision that comes after the seconds (those 6 decimals mentioned in point 1).
Unfortunately for me, I need that amount of precision in my system, so I can’t just cut out the last 6 decimals.
Questions
How can I fix my converter so it can properly override the Timestamp with the precision I need?
Fl4m3Ph03n1x
(Pedro Miguel Pereira Serrano Martins)
2
With the following converter:
yyyy-MM-dd'T'HH:mm:ss.SSSSSS
I am getting the following error:
java.lang.IllegalArgumentException: Invalid format: "2021-02-14T04:20:10.421162" is malformed at "162"`
at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945) ~[graylog.jar:?]
at org.joda.time.DateTime.parse(DateTime.java:160) ~[graylog.jar:?]
at org.graylog2.inputs.converters.DateConverter.convert(DateConverter.java:87) ~[graylog.jar:?]`
at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:260) [graylog.jar:?]
at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:247) [graylog.jar:?]
at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:77) [graylog.jar:?]
at org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.handleMessage(ProcessBufferProcessor.java:136) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:121) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:93) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
Which is surprising as S is a fraction of a second and according to DateTime it should be able to go up to 9 S:
What am I missing?
Fl4m3Ph03n1x
(Pedro Miguel Pereira Serrano Martins)
3
It is my belief that Graylog does not support Timestamps with microsecond precision. I was unable to confirm this however. Has anyone ever made a microsecond Timestamp work on Graylog?
I tried to look into this and have probably only confused myself more… I’m not a java developer but I was curious about this, so I looked at the source on github and I came across this issue:
So I was like… cool… it’s currently not, but they are looking to fix that. So I took a look at the
DateTime.Formats.TIMESTAMP_TZ format because it was DATETIME_TZ, and they changed it to TIMESTAMP_TZ. but when I tracked it down (again… not a Java guy) I see that every instance of DataTime is from the org.joda.time.DateTime library and from what I can tell, joda-time does not support microseconds.
Please take this with a grain of salt as again I am not a Java developer… perhaps @jan or @aaronsachs could shed some light on this.
