Display message with corrupt timestamp

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
Graylog reports this error while indexing

Invalid date graylog_1709 1dbcb0c2-c780-11ed-9f0d-0242ac110008 ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [timestamp] of type [date] in document with id ‘1dbcb0c2-c780-11ed-9f0d-0242ac110008’. Preview of field’s value: ‘292278994-08-17 07:12:55.807’]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=failed to parse date field [292278994-08-17 07:12:55.807] with format [uuuu-MM-dd HH:mm:ss.SSS]]]; nested: ElasticsearchException[Elasticsearch exception [type=date_time_parse_exception, reason=Text ‘292278994-08-17 07:12:55.807’ could not be parsed at index 0]];

2. Describe your environment:
Graylog 4.0.15
elasticsearch 7.10.2

  • OS Information:
    Linux 3.10.0-1127.19.1.el7.x86_64

  • Package Version:

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

I have tried searching the message in graylog by the document id.
I have tried fetching the message in directly in elasticserach by the document id

4. How can the community help?
We have an application that is sending gelf messages with an incorrect timestamp. I want to know if there is a way to find this message and display its contents so I can ascertain the application sending this message. My hope is there is way to dump all the messages in a given index and then I could search for the invalid timestamp and find the other fields of the message that would help me figure out which application sent the message. I haven’t found a way to do this yet.

Thanks in advance.

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

Hello && welcome @bwagner

There is no easy way about this, You could try the stream called Processing and Indexing Failures . If not then you maybe able to try gl2_message_id it build in so something like this in the search bar.

gl2_message_id: 1dbcb0c2-c780-11ed-9f0d-0242ac110008

If niether one does work you may have to break it down to INPUTs and index’s used. Normally windows with GELF are not an issues its firewall, switches, databases, etc… Sorry there is not a good easy find or fix.

3

Thanks for the advice. I will give it a shot. I was able to find the issue by writing my own gelf reader to capture traffic and search the results.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.