Set timestamp with pipeline

[BradBooth]

I’m having trouble setting the timestamp field of my logs with a different timestamp. I’ve seen the issue posted before here: Searching imported logs by log timestamp, not time Graylog received the log - #4 by yett but I can’t seem to get it to work.

Here is my current pipeline rule, I simply want to see if I am able to change the timestamp to a custom string (or date object).

rule "set timestamp" when true then let new_date = parse_date(to_string("2017-06-02T20:38:43.851Z"), "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"); set_field("timestamp", new_date); end

If I set a custom field to the “new_date” the format appears as follows:

Custom field
2017-06-02T20:38:43.851Z

But if I try to set the timestamp field, the log is dropped.
My timestamp field has the following format:

timestamp
2017-07-12T19:09:48.010Z

Thanks

[jochen]

The date format string is wrong.

The Z shouldn’t be in quotes (because that is a literal “Z”) but be part of the format string, i. e. “yyyy-MM-dd’T’HH:mm:ss.SSSZ”), see http://www.joda.org/joda-time/key_format.html for technical details.

[BradBooth]

@BradBooth

let new_date = parse_date(to_string(“2017-06-02T20:38:43.851Z”), “yyyy-MM-dd’T’HH:mm:ss.SSS’Z’”);

Ah you’re right, the Z should not have been in quotes.
I’m still having some issues however, perhaps I’m not understanding how this should work.
I’m trying to create a simple date object by using parse_date.

let new_date = parse_date("2015", "yyyy");

If I do this, then I can set a custom field using:

set_field("Custom Field", new_date);

And it appears on the logs as follows:

Custom Field
2015-01-01T00:00:00.000Z

But if I do the same thing but for the timestamp field, the log is dropped and it doesn’t show in Graylog.

set_field("timestamp", new_date);

Rule:

rule “set timestamp”
when
true
then
let new_date = parse_date(“2015”, “yyyy”);
set_field(“timestamp”, new_date);
end

Is there maybe a more specific format that my date needs to be in?

Thanks again for your help.

[BradBooth]

Well this is embarrassing. The log wasn’t being dropped, I simply wasn’t searching in the right time frame. The timestamp was being changed.