Incorrect time in log

I have create this pipeline

rule "timestamp_now+2"
when
true
then
let new_date = parse_date( to_string($message.timestamp),"yyyy-MM-dd HH:mm:ss.SSS","Europe/Paris");
set_field("timestamp", new_date);
end

And I’ve this error message from input :

For rule 'timestamp_now+2': In call to function 'parse_date' at 5:15 an exception was thrown: Invalid format: "2018-08-31T14:42:07.000+02:00" is malformed at "T14:42:07.000+02:00"

Do you have an idea ?

you give the rule the information that your timestamp is looking: yyyy-MM-dd HH:mm:ss.SSS but it is actually yyyy-MM-ddTHH:mm:ss.SSSX(should work this way)

Thanks for information.

I try to use this

When I try with your solution it’s doesn’t works. In fact I have this message in my logs :
For rule 'timestamp_now+2': In call to function 'parse_date' at 5:15 an exception was thrown: Illegal pattern component: T

When I delete the T I have this error message:
For rule 'timestamp_now+2': In call to function 'parse_date' at 5:15 an exception was thrown: Illegal pattern component: X

I try to use this pipeline :

rule "timestamp_now+2"
when
true
then
let new_date = parse_date( to_string($message.timestamp)," yyyy-MM-dTHH:mm:ss.SSSX","Europe/Paris");
set_field("timestamp", new_date);
end

Do you have an idea ?

I try to use an other element that I see on the forum :

let new_timestamp = parse_date(to_string($message.log_timestamp), “yyyy-MM-dd HH:mm:ss,SSS”, “AU”, “Australia/Perth”);

But It’s doesn’t works, I have a problem with this :
For rule 'timestamp_now+2': In call to function 'parse_date' at 5:20 an exception was thrown: Invalid format: ""

Do you have an idea ?

Thanks in advance

Sorry that I missed one Important in the time …

yyyy-MM-dd'T'HH:mm:ss.SSSX

You need to match the parse string 100% to your time - that is the reason for your errors

1 Like

I already tried to use it but I have this error message :

gl2_processing_error

For rule 'timestamp_now+2': In call to function 'parse_date' at 5:15 an exception was thrown: Illegal pattern component: X

Here is my pipeline configuration :

rule "timestamp_now+2"
when
true
then
let new_date = parse_date( to_string($message.timestamp),"yyyy-MM-dd'T'HH:mm:ss.SSSX","Europe/Paris");
set_field("timestamp", new_date);
end

When I try to use with Z in place of X, I have no error message the time don’t change.

When I try to use with Z in place of X, I have no error message the time don’t change.

And if you remove that?

I have this return :

For rule 'timestamp_now+2': In call to function 'parse_date' at 5:15 an exception was thrown: Invalid format: "2018-09-04T10:22:18.000+02:00" is malformed at "+02:00"

As you can see here: https://stackoverflow.com/questions/2375222/java-simpledateformat-for-time-zone-with-a-colon-separator

It should work with yyyy-MM-dd'T'HH:mm:ss.SSSZ with that is trial and error currently as I do not know how your local timestamp field look like.

The timestamp is that for the moment :

Capture

It’s not easy to use the pipelines. Because I have the good time in error message with the good timezone (Europe/Paris), but I have gl2_proccessing error :

Do you know what is the problem ?

Thanks

It’s possible to change the date format with good timezone ?

I still have the same problem, but I can not find a solution

Now I use this configuration on the pipeline :

rule "timestamp_now+2"
when
true
then
let new_date = parse_date( to_string($message.timestamp),"yyyy-MM-dd'T'HH:mm:ssZ","Europe/Paris");
set_field("timestamp", new_date);
end

I have this on my Input :

The time at the top left is the same as in the error message (gl2_processing_error), it has the correct time zone.

However, I still have this error and therefore the timestamp field does not take into account the correct time zone.

Do you have an idea ?

Thanks

Now I testing 2 new rules for change date :

rule "timestamp_now+2"
when
    has_field("message")
then
    let pattern = "%{GROK}";
    let matches = grok(pattern: pattern, value: to_string($message.message));
    set_fields(matches);
    let new_date = parse_date(to_string($message.timestamp),"MMM dd HH:mm:ss","Europe/Paris");
    set_field("timestamp", new_date);
end

And this :

rule "timestamp_now+2"
when
true
then
let current_year = now().year;
let ts_string = concat(to_string(current_year), concat(" ", to_string($message.timestamp)));
let new_date = parse_date(to_string($message.timestamp),"yyyy MMM dd HH:mm:ss","Europe/Paris");
set_field("timestamp", new_date);
end

But the 2 rules doesn’t works.

For the 1st I have this return :

And for second I have this return :

Do you have an idea, why this 2 rules doesn’t works ?

Thanks

Nothing have an idea ?

This problem of timestamp will be corrected in the next version of Graylog (3.0) ?

I have already an gl2_processing_error, when I try to change the datetime with the good timezone. The pipeline doesn’t work in Graylog.

Nothing have idea ?

Thanks

Hello,

I have the exact same problem.
I´m trying to correct the timestamp with the pipeline rules.
When I get the error message the time in error message appears changed to correct time zone +02:00.
When I haven´t got the error message, meaning everything´s right with the rule, the timestamp hasn´t been changed in the log (timestamp remains on UTC).

Here is my rule that doesn´t produce any errors but unfortunately also doesn´t changing the timestamp:

rule “timestamp change”
when
true
then
let new_date = parse_date(
value: to_string($message.timestamp),
pattern: “yyyy-MM-dd’T’HH:mm:ss.SSSZ”,
timezone: “Europe/Berlin”
);
set_field(“XXX”, new_date);
end

Does anyone has an idea?

Thanks

the information that is visible in the first screenshot might give you the suggestion that something is wrong - but let me explain.

When you expand the message and see the field timestamp Graylog will show the timestamp that is used to save the message - this will always be the time in UTC as Graylog use internal UTC.
The second timestamp that is shown left in the listed view is this timestamp translated to the timezone that the current active user has set in his profile. For the hardcoded admin this is whatever is written in “root_timezone” in your server.conf - where the default is UTC.