ElasticSearch Date Issue

[AaronFaby]

Hello all,

I have an interesting problem that I cannot seem to solve. I have some Palo Alto firewalls sending traffic log data to a Graylog instance, which was working fine for quite a while. I am using a Grok pattern to parse the data. A few fields have time data in them in the following format:

2017/04/04 13:39:34

Unfortunately this is no longer working and the following error is displayed from ElasticSearch:

MapperParsingException[failed to parse [panos_traffic_generated_time]]; nested: IllegalArgumentException[Invalid format: “2017/04/06 08:08:34” is malformed at “/04/06 08:08:34”];

Any ideas on how to resolve this? The log format from the Palo Alto has not changed.

Thanks!

[jochen]

If you always fill panos_traffic_generated_time with a valid date and do not want to use a plain string field for that, you should create a custom index mapping for this (and similar) fields:

• http://docs.graylog.org/en/2.2/pages/configuration/elasticsearch.html#custom-index-mappings
• https://www.elastic.co/guide/en/elasticsearch/reference/2.4/date.html
• https://www.elastic.co/guide/en/elasticsearch/reference/2.4/mapping-date-format.html

[AaronFaby]

Thanks, but I’m not much of an ElasticSearch expert.

I ended up disabling those fields as I don’t even need them, but I’m trying to figure out why they randomly stopped working.