I want to send a date via GELF to Graylog. What format should I use for the date value so that it is directly interpreted as a date and time, and displayed in the UI as a date and time (not a number value)?
The mandatory timestamp parameter in each GELF message is to be a floating point value. If I want to send another timestamp value, should I use the same floating point format? (convert to UTC, then UNIX format.) I tried this and it simply shows the value in the UI as a floating point value. Is there a better format I should be using?
I’m currently sending timestamps as “2020-06-01T06:00:00.0000000”
When I click on the parameter in the web ui message view, Graylog reports it as a “compound(date,string)”.
This seems to indicate that Graylog isn’t consuming it as a timestamp that includes date and time. While I’m able to search by date range, it doesn’t seem to recognize the time as well. I have full control of the value being sent, so I can format it however I want. So, my question is simply whether there is a particular timestamp format that is directly consumable by Graylog so that it will recognize the full date+time value. And, is there a place in the documentation I’ve missed that would give information about this? I’d certainly be willing to read up on it, if I’ve missed it.
Seconds since UNIX epoch with optional decimal places for milliseconds; SHOULD be set by client library. Will be set to the current timestamp (now) by the server if absent.
Thanks shoothub. And yes, I read that specification in the documentation. But I’m not asking about the “timestamp” field. I’m asking about representing a datetime for a user defined field (in this case, “_install_datetime”). I had originally tried an epoch long value, and it was interpreted and displayed as a long value, rather than a date – which is what spurred me to post my question here.
rule “unix to date”
when
has_field(“install_datetime”)
then
let new_date = parse_unix_milliseconds(to_long($message.install_datetime);
debug(concat("Install datetime: ", to_string(new_date)));
set_field(“install_timestamp”, new_date);
end
Or send as date string and then parse using pipeline rule:
For example for ‘2010-07-30T18:03:25+02:00’ you can use this pipeline rule:
rule “parse date”
when
has_field(“install_datetime”)
then
let new_date = parse_date(to_string($message.install_datetime), “yyyy-MM-dd’T’HH:mm:ssZZ”));
set_field(“install_datetime”, new_date);
end
Thanks again shoothub. Yes, a pipeline would work to convert the incoming value.
But, based on your answer, am I understanding correctly that the answer to my question is that there is no format that Graylog can consume directly to date, and a Pipeline rule is necessary to save the value as a date or timestamp value?
When I click on the parameter in the web ui message view, Graylog reports it as a “compound(date,string)”.
This indicate that you have the same field name in different indices as date OR string. Means over the time of searching you have some indices that holds a date and some that holds a string.
This seems to indicate that Graylog isn’t consuming it as a timestamp that includes date and time. While I’m able to search by date range, it doesn’t seem to recognize the time as well. I have full control of the value being sent, so I can format it however I want. So, my question is simply whether there is a particular timestamp format that is directly consumable by Graylog so that it will recognize the full date+time value. And, is there a place in the documentation I’ve missed that would give information about this? I’d certainly be willing to read up on it, if I’ve missed it.
If the time is send consistence in a field as date (as you did) and you do not have other messages that use the same field name but send strings this will happen. What you could always do is create a custom mapping for your indices in elasticsearch to force this field to be a date field and as long as the date is something that elasticsearch can ingest the message will be accepted, otherwise it would be dropped.
