Yet another mapper_parsing_exception

[JulioQc (Julien)]

So I keep getting those indexing failures very often:

{"type":"mapper_parsing_exception","reason":"failed to parse [date-heure]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"#Fields: date-time\""}}

The extractor for my Exchange 2016 logs is a CSV to field as:

date-heure,connector-id,session-id,sequence-number,local-endpoint,remote-endpoint,event,data,context

An actual corresponding (obfuscated) log line would be:

2018-11-14T15:04:27.478Z,MAIL-xx-M\Client Proxy MAIL-xx-M,08D63CBEF254F356,1,192.168.xx.xx:465,192.168.xx.xx:47907,>,“220 MAIL-xx-M.xxxx.ca Microsoft ESMTP MAIL Service ready at Wed, 14 Nov 2018 10:04:26 -0500”,

So what is failing here? How to investigate since I can’t view the message that actually failed?

[jtkarvo]

Try:
https://www.elastic.co/guide/en/elasticsearch/reference/5.2/indices-get-field-mapping.html

something like (replace with the right address and index number

curl -XGET http://my.es.server:9200/graylog_XXX/_mapping/field/date-heure

… although: to me it seems your extractor tries to extract the header line, where obviously the word date-time is not in a proper date-format. You could try first dropping all lines starting with #, and only after that applying the extractor

[JulioQc (Julien)]

Yes… good call! the log file headers are what’s causing issues here! Especially that the failures occurs every hour when the logs rotate.

I’ll adjust my regex and test! Thank you

[JulioQc (Julien)]

Yup, was headers breaking the parsing. Fixed by excluding lines starting with # symbol. Thanks mate!