ERROR [Extractor] Could not apply converter [DATE] of extractor

[ssstonebraker (Steve Stonebraker)]

Greetings. I am seeing errors in my log file even though my extractor is properly converting epoch dates. I’m wondering why the errors are triggering (and how to stop them).

Environment:

• Graylog 3.1.4+1149fe1
• Elastic Search 5.6

Error message:

2021-03-23T16:42:09.404-05:00 ERROR [Extractor] Could not apply converter [DATE] of extractor [9f96c3c4-6aee-11ea-ab2c-0efa831ef056].
java.lang.IllegalArgumentException: Invalid format: "1616510609950" is malformed at "9950"
	at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945) ~[graylog.jar:?]
	at org.joda.time.DateTime.parse(DateTime.java:160) ~[graylog.jar:?]
	at org.graylog2.inputs.converters.DateConverter.convert(DateConverter.java:87) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:260) [graylog.jar:?]
	at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:247) [graylog.jar:?]
	at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:77) [graylog.jar:?]
	at org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterC^ChainProcessor.java:100) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.handleMessage(ProcessBufferProcessor.java:126) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:112) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:89) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:45) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

Example source message:

{"IntegrityLevel":"16384","ParentProcessId":"794440353170","SourceProcessId":"794440353170","aip":"67.185.87.215","SHA1HashData":"0000000000000000000000000000000000000000","UserSid":"S-1-5-18","event_platform":"Win","TokenType":"1","ProcessEndTime":"","AuthenticodeHashData":"32edce500c8f918b255a3d2b75a69d6a380006befcbc141b3f7ba8db9f134673","ParentBaseFileName":"ngentask.exe","ImageSubsystem":"3","id":"225278d1-8be6-11eb-b7b3-028d8a0523db","EffectiveTransmissionClass":"3","SessionId":"0","Tags":"53, 54, 12094627905582, 12094627906234","timestamp":"1616510609950","event_simpleName":"ProcessRollup2","RawProcessId":"27080","ConfigStateHash":"1390867394","MD5HashData":"b6c3fe33b436e5006514403824f17c66","SHA256HashData":"a446d35f5e60bdae1a7117b71d90abb0fea424ee416b0e0ecd2c485dc5939a68","AuthenticationId":"999","ConfigBuild":"1007.3.0012806.1","CommandLine":"\"C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319\\ngen.exe\" uninstall \"C:\\WINDOWS\\assembly\\NativeImages_v2.0.50727_64\\Microsoft.Ink\\4b1202b64c06bf0e2ef81f3f537923b1\\Microsoft.Ink.ni.dll\" /noroot /LegacyServiceBehavior","ParentAuthenticationId":"999","TargetProcessId":"794541773446","ImageFileName":"\\Device\\HarddiskVolume4\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319\\ngen.exe","SourceThreadId":"32172102601597","Entitlements":"15","name":"ProcessRollup2V19","ProcessStartTime":"1616505105.549","aid":"8ac8d6d48a78498b9bbc55bf9cbbddaf","SignInfoFlags":"1835008","cid":"6b4eb5e8aa1c41ea82159a331fbf8af5"}

Regular Expression:

\btimestamp\"\:\"([0-9]*)"

Extractor Preview:

1616510609950

Screenshot of timestamp_epoch being properly parsed and “timestamp” field being properly populated:

Screenshot of extractor config:

Screenshot of errors:

Please let me know if you need any additional information to debug. Thank you so much!

[shoothub]

That’s very obvious. Value 1616510609950 is epoch in ms, but graylog can’t parse it using extractor. So use pipeline rule like this:

rule "Epoch Convert"
when
  contains(to_string($message.message), "event_simpleName")
then
  let ts = regex(pattern: "\btimestamp\":\"([0-9]*)", value: to_string($message.message));
  let ts_millis = to_long(ts["0"]);
  let new_date = parse_unix_milliseconds(ts_millis, "America/Chicago");
  set_field("timestamp_epoch", new_date);
end

https://docs.graylog.org/en/4.0/pages/pipelines/functions.html#parse-unix-milliseconds

[ssstonebraker (Steve Stonebraker)]

Thank for responding. I tried your suggestion and end up with the following:
timestamp_epoch:

1970-01-01T00:00:00.000Z

What we have now that is working (except causes all the errors) is a pipeline rule and extractor.

The pipeline rule references the field timestamp_epoch created by the extractor

Extractor

 {
      "title": "CrowdStrike-Timestamp",
      "extractor_type": "regex",
      "converters": [
        {
          "type": "date",
          "config": {
            "locale": "en-US",
            "time_zone": "America/Chicago",
            "date_format": "yyyy-MM-dd HH:mm:ss.SSS Z"
          }
        }
      ],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "timestamp_epoch",
      "extractor_config": {
        "regex_value": "\\btimestamp\\\"\\:\\\"([0-9]*)\""
      },
      "condition_type": "string",
      "condition_value": "event_simpleName"
    }

Pipeline rule

rule "Crowdstrike-timestamp"
when
   to_string($message.filebeat_host_name) == "security-cstrike"
then
    let timestamp = parse_unix_milliseconds(to_long($message.timestamp_epoch));
    set_field("timestamp", timestamp);
end

Actual result
Timestamp extracts properly but my log is spammed with errors

Expected Result
Timestamp extracts properly