Elasticsearch parse exception/Indexer failures

I’ve the following error:

ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [apache_timestamp] of type [date] in document with id 'a0511820-1532-11ec-a2d4-0050568c8ce4'. Preview of field's value: 'Tue Sep 14 10:06:09.999727 2021']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=failed to parse date field [Tue Sep 14 10:06:09.999727 2021] with format [strict_date_optional_time||epoch_millis]]]; nested: ElasticsearchException[Elasticsearch exception [type=date_time_parse_exception, reason=Failed to parse with all enclosed parsers]];

does that mean, Graylog/Elasticsearch dropped the hole message? How can I figure out which server are sending logs? (In my opinion its a bug that a Log-Managementsystem is dropping(!!) logs)

Hello,

What is your environment, meaning what is your configuration to ingest logs and how you sending them?
As for your error this looks like elasticsearch expects the [apache_timestamp] field to be a date. Either you have a mapping misconfiguration, or it could be a generic ISO datetime parser, where the date must include the year at a minimum, and the time (separated by T), is optional. Examples: yyyy-MM-dd’T’HH:mm:ss.SSSZ or yyyy-MM-dd."

The date parser defaults to strict_date_optional_time or epoch_millis.
Did you create a template for mapping?

Edit: I did a quick google maybe this might help.

I send logs normally via filebeat, but some “older” webserver have strange configs with something lke |nc syslog.example.de:5514 so and it is not that easy to find them all… it would be much easier to know which IP/Source it was. it is also bad, that graylog/elastic drops this messages (for me this is a bug)

Hello,

I feel your frustration on logs dropped. In my lab I’m testing out Elastic-stack, Open Distro For ElasticsearchAWS Opensearch. Depending on the device and what logs are shipped, I run into the same problems with each of them. Sorry I cant be more help.

This is something we’re aware of and have a couple of issues open to address. The ideal approach is that we implement dead letter queues for elastic where we take the parts of a message that can’t be parsed and drop them

1 Like