Error in graylog when timestamp field data appending timezone by sending via logstash

Graylog Central (peer support)
1

Hi All,

We have built customized beats for our environment to get monitor data and data will be sending to beat port(5044) in graylog.

Once the data is received in beats input we are getting the below error in graylog server.log,

 2019-07-10T18:20:24.279Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=62826290-a33f-11e9-a847-0242579a69fb, journalOffset=24249, codec=beats, payloadSize=581, timestamp=2019-07-10T18:20:24.249Z, remoteAddress=/104.500.148.156:40820} on input <5d258efddbd26c04a8c32b55>.
 2019-07-10T18:20:24.280Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=628289a0-a33f-11e9-a847-0242579a69fb, journalOffset=24250, codec=beats, payloadSize=581, timestamp=2019-07-10T18:20:24.250Z, remoteAddress=/104.340.148.196:40820} on input <5d258efddbd26c04a8c32b55>.
 2019-07-10T18:20:24.281Z ERROR [DecodingProcessor] Error processing message RawMessage{id=628289a0-a33f-11e9-a847-0242579a69fb, journalOffset=24250, codec=beats, payloadSize=581, timestamp=2019-07-10T18:20:24.250Z, remoteAddress=/104.315.148.186:40820}
        java.lang.IllegalArgumentException: Invalid format: "2019-07-10T23:43:27+0530" is malformed at "+0530"
                at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945) ~[graylog.jar:?]
                at org.graylog2.plugin.Tools.dateTimeFromString(Tools.java:337) ~[graylog.jar:?]
                at org.graylog.plugins.beats.Beats2Codec.parseEvent(Beats2Codec.java:86) ~[graylog.jar:?]
                at org.graylog.plugins.beats.Beats2Codec.decode(Beats2Codec.java:78) ~[graylog.jar:?]
                at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
                at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
                at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
                at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
                at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
                at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
                at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

Not sure how the timezone is appending to the date to the timestamp field.

Please kindly share your thoughts…

Regards,
Ganeshbabu R

2

Where are you adding the offset?
It looks like it’s receiving with the Z at the end which would be invalid with Z+0530.

However I am just taking a wild guess here.

3

the timestamp should be given in UTC (Zulu) and not with a timezone.

4

Hi @jan

Yes but the same data which i am trying to send via logstash port (5044) running in instance not in Graylog inputs and that time I can see the data is saved without any issues,

Not sure how the logstash avoids the timezone when indexing to Elastic. But I am getting error when I am sending data to beats port via graylog.

Below is the sample data from elastic index.

  "host": {
    "name": "localhost"
  },
  "SSLTime": 693,
  "FirstByteTime": 208,
  "Counter": 5006,
  "Type": "AggResponseTime",
  "ConnectionTime": 207,
  "DownloadTime": 1247,
  "MonitorName": "testdata",
  "tags": [
    "beats_input_raw_event"
  ],
  "@version": "1",
  "@timestamp": "2019-07-01T00:20:19.000Z",
  "DNSTime": 0,
  "agent": {
    "type": "testbeat",
    "ephemeral_id": "d77d5cd8-e64e-4926-858a-8fcb5e9c6085",
    "id": "2fa762c8-7096-4024-8ced-f4b8c92021eb",
    "version": "8.0.0",
    "hostname": "hostname"
  },
  "MonitorID": "10000"

Please let me know your thoughts…

Regards,
Ganeshbabu R

5

Not sure how the logstash avoids the timezone when indexing to Elastic.

Maybe logstash converts the Timestamp - But I do not know that. That is just guessing!

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.