Pipeline for timestamp

Graylog Central (peer support)
1

Hi Team,

I have a time field in the log as “Feb 19 18:50:03”, how do i write a pipeline to set this time as the timestamp.

Regards,
Vinay.

2

You can parse the string with parse_date() and assign the result to the “timestamp” message field with set_field().

3

Hi jochen,

I am using he pipeline rule:

rule "XXX"
when
    has_field("message")
then
    let pattern = "%{GROK}";
    let matches = grok(pattern: pattern, value: to_string($message.message));
    set_fields(matches);
    let new_date = parse_date(to_string($message.timestamp),"MMM dd HH:mm:ss","Asia/Kolkata");
    set_field("timestamp", new_date);
end

When i simulate the it, i get the below as new timestamp:

2000-02-19T18:50:03.000Z

It is taking “2000” (year) , but it is not correct, it should take current year ie 2018.
How do i achieve this?

Regards,
Vinay.

4

PS. the log dose not has a year. All it has is “Feb 19 18:50:03”.

5

It’s a bit of a hacky workaround, but you could prepend the current year to the field so that the date parser can use it.

Example:

let current_year = now().year;
let ts_string = concat(to_string(current_year), concat(" ", to_string($message.timestamp)));
let new_date = parse_date(to_string($message.timestamp),"yyyy MMM dd HH:mm:ss","Asia/Kolkata");
set_field("timestamp", new_date);

But beware of edge cases such as new years!

Please also create a feature request at https://github.com/Graylog2/graylog2-server/issues to make it possible to specify the default year (and other properties, such as default month) in the parse_date() function or the DateTime data type in the processing pipelines.

6

Hi Jochen,

Thank you. This worked. :slight_smile:

Regards,
Vinay.