Handling comma when converting extracted data to date type

draganHR · April 19, 2018, 1:17pm

Hi,

I am trying to convert data type of timestamp extracted with grok pattern, as described in documentation:

%{DATA:timestamp;date;dd/MMM/yyyy:HH:mm:ss Z}

Formatted time looks like this: 2018-04-19 13:01:24,450
I am using TIMESTAMP_ISO8601 to extract this data, and if i do this, it does work:

%{TIMESTAMP_ISO8601:timestamp;date;yyyy-MM-dd HH:mm:ss}

But i’m losing miliseconds. If i add miliseconds, like so:

%{TIMESTAMP_ISO8601:timestamp;date;yyyy-MM-dd HH:mm:ss,SSS}

…i get this error:

We were not able to run the grok extraction. Please check your parameters.
Details: Error: cannot POST http://hostname:9000/api/tools/grok_tester (500)

Error response looks like this:

{"type":"ApiError","message":"Illegal repetition near index ...

It seems , character inside SimpleDateFormat expression yyyy-MM-dd HH:mm:ss,SSS is the issue. Are there any workarounds? I tried escaping it with \ but it didn’t help.

jochen · April 19, 2018, 1:23pm

Which version of Graylog are you using?

A similar date pattern is being used as test case in Graylog:

github.com

Graylog2/graylog2-server/blob/2.4.3/graylog2-server/src/test/java/org/graylog2/inputs/extractors/GrokExtractorTest.java#L70


    final GrokExtractor extractor = makeExtractor("%{NUMBER:number;int}");


    final Extractor.Result[] results = extractor.run("199999");
    assertEquals("NUMBER is marked as UNWANTED and does not generate a field", 1, results.length);
    assertEquals(Integer.class, results[0].getValue().getClass());
    assertEquals(199999, results[0].getValue());
}


@Test
public void testDateExtraction() {
    final GrokExtractor extractor = makeExtractor("%{GREEDY:timestamp;date;yyyy-MM-dd'T'HH:mm:ss.SSSX}");
    final Extractor.Result[] results = extractor.run("2015-07-31T10:05:36.773Z");
    assertEquals("ISO date is parsed", 1, results.length);
    Object value = results[0].getValue();
    assertTrue(value instanceof Date);
    DateTime date = new DateTime(value, DateTimeZone.UTC);


    assertEquals(2015, date.getYear());
    assertEquals(7, date.getMonthOfYear());
    assertEquals(31, date.getDayOfMonth());
    assertEquals(10, date.getHourOfDay());

draganHR · April 19, 2018, 1:44pm

Version is 2.4.3+2c41897.
Thanks for reply, pattern in that use case would work fine if my time format was different. I can use dot in the expression, but comma fails.

jochen · April 19, 2018, 2:16pm

It looks like this is a bug in the Grok library which is being used by Graylog:

mev9669 · April 19, 2018, 2:42pm

Try to debug your GROK Pattern using different inputs
[https://grokdebug.herokuapp.com/](http://Grok Debugger)

system · May 3, 2018, 2:42pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Parse milliseconds timestamp strange behavior(java.time.format.DateTimeParseException) Graylog Central (peer support)	3	1170	November 10, 2020
GROK Pattern Skipping Data Graylog Central (peer support)	4	2179	September 14, 2020
Problem with date extractor converter Graylog Central (peer support)	5	2237	June 28, 2022
How do I upgrade the java-grok graylog jar? Graylog Central (peer support) pipeline-rules , time-stamp-issuespl	10	1370	July 5, 2019
Extract date from message Graylog Central (peer support)	3	5042	April 21, 2017

Handling comma when converting extracted data to date type

Related topics