Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!
1. Describe your incident:
We are receiving processing errors for 2 different log sources relating to timestamps.
Failure Details Example:
Value <2024-08-22T17:55:22.875418Z> caused exception: Invalid format: “2024-08-22T17:55:22.875418Z” is malformed at “T17:55:22.875418Z”.
2. Describe your environment:
2 node cluster in Graylog Cloud of 6.0.4
3. What steps have you already taken to try and solve the problem?
I have attempted to do research onto how other people have solved this issue.
I found pipeline rules extracting/parsing the timestamp to be the solution for most people but its just not working for me.
Here is the current pipeline rule I am trying out, doesn’t seem to work:
rule "testing timestamps"
when
true
then
let result = regex("([0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3})",to_string($message.message));
let new_date = parse_date(to_string(result["0"]), "yyyy-MM-dd'T'HH:mm:ss.SSS","EST");
set_field("timestamp", new_date);
end
This doesn’t appear to set the new date correctly:
The regex appears to be good.
I have also played around with the date format in new_date trying different variations. Where am I going wrong? For what its worth, I’m pretty new to pipeline rules so it might be painfully obvious what I’m doing wrong to you guys.
4. How can the community help?
Looking for a bit of guidance on next steps.