Parse unixtime with milliseconds

Recently I found one weird thing about how Graylog/ES handle Nginx access logs(sending to Graylog via syslog)
It seems that if messages have the same timestamp(i.e. 04/10/2019 01:02:03) messages order can be broken during handling in Graylog and writing to ES.
So when I query it in Graylog I can see message that logically comes first(it confirmed by app logs) is displayed as second or third.
I’m trying to workaround this by adding milliseconds to Nginx timestamps.
Nginx provides $msec variable, which is actually represented as unixtime with milliseconds.(Example:1407233265.472)
The question is how I can efficiently proceed such timestamp in Graylog?
Is there some solution like for Logstash ?

There is a pipeline function parse_unix_milliseconds that you can use to do this :slight_smile: (see also:

Please confirm my understanding: you suggest to convert it to milliseconds first and then use parse_unix_milliseconds

I’m pretty sure it will accept the timestamp as it is right now :slight_smile: One way to find out: just try it :smiley:

I’ve checked and nope, it doesn’t work like you expected

Aw. Damn. Okay, well, euhh… then there’s 2 solutions :smiley: Either configure nginx to send a timestamp in ISO8601 format (yyyy-mm-dd hh:mm:ss.SSSZ) or take the value that’s there, multiply by 1000 (I think you can do this in pipeline) and that may solve it :smiley:

So far I just remove decimal point

rule "Test milliseconds"
    let test = regex(pattern: ".*msec=(\\d{10}\\.\\d{3})", value: to_string($message.message));
    let timestamp = replace(to_string(test["0"]), ".", "");
    let timestamp2 = parse_unix_milliseconds(to_long(timestamp));
    debug(concat("Result: ", to_string(timestamp2)));

Any improvement suggestions will be much appreciated

Unfortunately $time_iso8601 doesn’t contain milliseconds: 2019-04-10T20:26:18+07:00 :anguished:

Possible implementation on Nginx side - extract ms and concatenate it with standard timestamp:

map $msec $go {
    "~*\d{10}(\.\d{3})" "$1";
map $time_local $tmp_time {
    "~(?<dt>\d{2}\/...\/\d{4}:\d{2}:\d{2}:\d{2}) (?<tz>\+\d{4})"    "$dt$go $tz";

Then use $tmp_time in the log

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.