Date Processing

[danmassa7]

I’m working with rules in the pipelines. I’m trying to get the proper date into Graylog based on the date in the syslog message.
My Aruba messages are similar to this:
<139>Feb 3 10:12:24 2022 mobility01 authmgr[3950]: <132197> <ERRS> <mobility01 192.0.2.5> Maximum number of retries was attempted for station 8c:fc:5a:92:53:d4

I can grok out the pieces:

    let extract = grok(
        pattern: "<%{NONNEGINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_time} %{YEAR:year}",
        value: to_string($message.message),
        only_named_captures: true
        );

But what do I do with this now? I can create a unified field to process:

    let message_time_str = concat(to_string(extract.syslog_time), " ");
    let message_time_str = concat(to_string(message_time_str), to_string(extract.year));

That gives me a field of the form:
Feb 3 10:12:24 2022

Great. But where do I go from here?

If I try to run it through parse_date()…
let date_time_object = parse_date(message_time_str, "MM dd hh:mm:ss yyyy");
…it fails with:

gl2_processing_error
`For rule 'raw-process-exp': In call to function 'parse_date' at 51:27 an exception was thrown: Invalid format: "Feb  3 10:12:24 2022"`

I clearly don’t know how to use the parse_date() function!

Thanks.

Operating system information

centos-linux-release-8.3-1.2011.el8.noarch

Package versions

• Graylog 4.0.7
• MongoDB 4.2.14
• Elasticsearch 7.10.2

[gsmith]

Hello,

@danmassa7

parse_date() function!

Perhaps something like this?

rule “parse date”
when
    has_field(“some_field”)
then
    let new_date = parse_date(to_string($message.some_field), “yyyy-MM-dd’T’HH:mm:ssZZ”));
    set_field(“install_datetime”, new_date);
end

Don’t know if you seen this?

https://docs.graylog.org/docs/functions

Hope that helps

[danmassa7]

I found a reference that goes into more detail and has examples (the examples are key!)…

Supported date formats - Amazon QuickSight

[danmassa7]

Here’s where I landed. Thank you @gsmith for your help.

    let message_time_str = concat(to_string(extract_1.message_timestamp), to_string(extract_1.year));
    let date_time_object = parse_date(
        value: to_string(message_time_str), 
        pattern: "MMM dd HH:mm:ssYYYY",
        timezone: "US/Eastern"
        );
        
    set_field("timestamp", date_time_object);

This process gets the year in there and sets the timezone correctly. Finally, it overwrites the timestamp field that used to hold the timestamp of when Graylog received the message (the default) but I want it to hold the timestamp that was put into the original syslog message.

I find I’m frequently using the to_string() function and things break down when I don’t use it. Doesn’t it already know it’s a string!? I grew up using Pascal and C++. How the system doesn’t already know it’s a string is beyond me.