Date Processing

I’m working with rules in the pipelines. I’m trying to get the proper date into Graylog based on the date in the syslog message.
My Aruba messages are similar to this:
<139>Feb 3 10:12:24 2022 mobility01 authmgr[3950]: <132197> <ERRS> <mobility01> Maximum number of retries was attempted for station 8c:fc:5a:92:53:d4

I can grok out the pieces:

    let extract = grok(
        pattern: "<%{NONNEGINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_time} %{YEAR:year}",
        value: to_string($message.message),
        only_named_captures: true

But what do I do with this now? I can create a unified field to process:

    let message_time_str = concat(to_string(extract.syslog_time), " ");
    let message_time_str = concat(to_string(message_time_str), to_string(extract.year));

That gives me a field of the form:
Feb 3 10:12:24 2022

Great. But where do I go from here?

If I try to run it through parse_date()…
let date_time_object = parse_date(message_time_str, "MM dd hh:mm:ss yyyy");
…it fails with:

`For rule 'raw-process-exp': In call to function 'parse_date' at 51:27 an exception was thrown: Invalid format: "Feb  3 10:12:24 2022"`

I clearly don’t know how to use the parse_date() function!


Operating system information


Package versions

  • Graylog 4.0.7
  • MongoDB 4.2.14
  • Elasticsearch 7.10.2


Perhaps something like this?

rule “parse date”
    let new_date = parse_date(to_string($message.some_field), “yyyy-MM-dd’T’HH:mm:ssZZ”));
    set_field(“install_datetime”, new_date);

Don’t know if you seen this?

Hope that helps

I found a reference that goes into more detail and has examples (the examples are key!)…

Here’s where I landed. Thank you @gsmith for your help.

    let message_time_str = concat(to_string(extract_1.message_timestamp), to_string(extract_1.year));
    let date_time_object = parse_date(
        value: to_string(message_time_str), 
        pattern: "MMM dd HH:mm:ssYYYY",
        timezone: "US/Eastern"
    set_field("timestamp", date_time_object);

This process gets the year in there and sets the timezone correctly. Finally, it overwrites the timestamp field that used to hold the timestamp of when Graylog received the message (the default) but I want it to hold the timestamp that was put into the original syslog message.

I find I’m frequently using the to_string() function and things break down when I don’t use it. Doesn’t it already know it’s a string!? I grew up using Pascal and C++. How the system doesn’t already know it’s a string is beyond me. :wink:

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.