Timestamp warning when sending GELF from Rsyslog

Graylog Central (peer support)
1

Hi, all!

I’m trying to send GELF messages from rsyslog (version 8.16.0-1ubuntu3 on Ubuntu 16.04.4 LTS) to Graylog (version 2.3.1-1) with this template:

template(name="gelf_docker" type="list") {
    constant(value="{\"version\":\"1.1\",")
    constant(value="\"host\":\"")
    property(name="hostname")
    constant(value="\",\"short_message\":\"")
    property(name="msg" format="json")
    constant(value="\",\"timestamp\":\"")
    property(name="timegenerated" dateformat="unixtimestamp")
    constant(value="\",\"_application_name\":\"kubernetes\",")
    constant(value="\"_full_container_name\":\"")
    property(name="$.full_container_name")
    constant(value="\",\"_container_name\":\"")
    property(name="$.container_name")
    constant(value="\",\"_container_id\":\"")
    property(name="$.container_id")
    constant(value="\",\"_pod_name\":\"")
    property(name="$.io_kubernetes_pod_name")
    constant(value="\",\"_pod_namespace\":\"")
    property(name="$.io_kubernetes_pod_namespace")
    constant(value="\"}\n")
}

But, I’m getting a lot of warnings on Graylog server log like this:

2018-07-29T00:37:21.389+03:00 WARN  [GelfCodec] GELF message <68bb24af-92ae-11e8-9967-005056b51dfc> (received from <:53874>) has invalid "timestamp": 1532813840  (type: STRING)

Rsyslog template was made according to this manual: Gelf forwarding

2

the timestamp need to be ISO8601 not Unix.

3

Hi!

According to this doc: Rsyslog property replacer rsyslog doesn’t have ISO8601 option for data format, but has rfc3339 option. I’ve tried to change line with dateformat to:

property(name="timegenerated" dateformat="rfc3339")

But it didn’t help, now i have such warnings in Graylog server log:

2018-08-10T11:01:56.324+03:00 WARN [GelfCodec] GELF message <a67d9428-9c73-11e8-9967-005056b51dfc> (received from <:43631>) has invalid "timestamp": 2018-08-10T11:01:55.979122+03:00 (type: STRING)

4

my bad - according to the spec unixtimestamp should work.

http://docs.graylog.org/en/2.4/pages/gelf.html#gelf-payload-specification

This might be something with your Graylog Version (Current stable 2.4.6) but not sure. About.

5

@pvlltvk did you get anywhere with this particular WARN message? I’m seeing the exact same thing (lots of identical warnings) in v2.4.6. Are you running 2.4.6 as well?

2018-08-23T14:51:21.478-05:00 WARN  [GelfCodec] GELF message <e8ab8b50-a70d-11e8-8c3f-0050569991aa> (received from <10.10.10.22:52047>) has invalid "timestamp": 1535053881.403  (type: STRING)
2018-08-23T14:51:24.560-05:00 WARN  [GelfCodec] GELF message <ea81f900-a70d-11e8-8c3f-0050569991aa> (received from <10.10.10.22:52047>) has invalid "timestamp": 1535053884.535  (type: STRING)
2018-08-23T14:51:24.744-05:00 WARN  [GelfCodec] GELF message <ea9de570-a70d-11e8-8c3f-0050569991aa> (received from <10.10.10.22:52047>) has invalid "timestamp": 1535053884.72  (type: STRING)
2018-08-23T14:51:24.801-05:00 WARN  [GelfCodec] GELF message <eaa69800-a70d-11e8-8c3f-0050569991aa> (received from <10.10.10.22:52047>) has invalid "timestamp": 1535053884.776  (type: STRING)
2018-08-23T14:51:24.802-05:00 WARN  [GelfCodec] GELF message <eaa6bf11-a70d-11e8-8c3f-0050569991aa> (received from <10.10.10.22:52047>) has invalid "timestamp": 1535053884.776  (type: STRING)
6

@MrMike

you could open a bug issue for that: https://github.com/Graylog2/graylog2-server/issues

It looks like the timestamp is not allowed to contain microseconds what is causing the issue.

7

No, i’m still on 2.3.1, but now i’m thinking, that it’s not version of specific bug :smiley:
By the way, I’ve tried various timestamps: unixtime (with or without microseconds), rfc3389, but warning messages still remains.