Extract date from message

Graylog Central (peer support)
1

Hi all.

I’m trying to extract a date from the log message and store on field “timestamp”, as described here (http://docs.graylog.org/en/2.1/pages/extractors.html#the-standard-date-converter). The problem is the timestamp keeps the value when the message arrived to graylog server. There are no errors on graylog server logs.

I’m using graylog 2.1.2, and Java 1.8.0_121 and elasticsearch 2.3.1.

Any idea?
Thanks.

2

Please be more specific.

  • What extractors and converters are you using?
  • What messages do you receive and how should they be transformed?
3

Hi Jochen.

Here’s an example of a received message. All the messages follow the same pattern.

[2017-04-21 13:38:11,957][INFO ][node   ][graylog_es_node_smartiot-big] initialized..

I’ve configured an extractor on a RAW TCP input. Here’s the details:

Extractor type: Regular expression
Regular expression: ^[(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})
Condition: Always try to extract
Store as field: timestamp
Extraction strategy: Copy

Converters:
Convert to date type.
Format string: yyyy-MM-dd HH:mm:ss,SSS
Time Zone:

I’m not sure what’s going on here! There’s another input with the same type of converter and it’s working. The only difference is that the date contains the time zone (Z). I’ve tried to set the “Time Zone” to UTC on this converter, but no success. Storing the conversion in a separate field works as expected. Storing the conversion in “timestamp” field seems to have an unpredictable behaviour…:-/

Does graylog definitely support timestamp field override?

Thanks.

4

[ and ] are reserved characters in regular expressions. If you want to match literal [ or ], you’ll have to escape these (\[ and \]).

You can use http://www.freeformatter.com/java-regex-tester.html to play around with your regular expressions.