Match Message Against a timestamp RegEx

Graylog Central (peer support)
1

I am a beginner and getting acquainted with GrayLog features.
I have an incoming stream of messages in format that starts with “[2021-05-12T13:01:11.123]”, I can match this sequence with expression: ([0-9-T.:]+). I want to replace the timestamp in GrayLog with this matched string. I am creating a rule in a pipeline for this stream.

rule "replace timestamp"
when
    true
then
    let result = regex("([0-9-T.:]+)", $message);
    let new_date = parse_date(result, "yyyy-MM-dd'T'HH:mm:ss","IST");
    set_field("timestamp", new_date);
end

It throws errors due to the regex expression when applying and saving.

2

Your pipeline rule have some problems:

  1. You use wrong $message definition, try use to_string($message.message)
  2. You need to use index or name or group to point to regex result.
  1. Your format in parse_date function missing milliseconds
    Functions — Graylog 4.1.0 documentation

So try this one:

rule "replace timestamp"
when
    true
then
    let result = regex("([0-9-T.:]+)", to_string($message.message));
    let new_date = parse_date(to_string(result["0"]), "yyyy-MM-dd'T'HH:mm:ss.SSS","IST");
    set_field("timestamp", new_date);
end
1 Like
3

Thanks a lot for such a comprehensive answer, that too with exact corrections. Looks like I am still not used to reading documentation properly yet.

Also, I had to convert result["0"] to to_string(result["0"]), in case someone stumbles upon this question in future.

4

Hi @abigdumbNerd
thanks for correction, I’ve fixed from my memory. I’ve updated fixed pipeline to be functional.

1 Like
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.