Elastic Search Unclear Mapper Parsing Exception

[optimaX]

Incident:
I’m currently working on a log pipeline which forwards the logs from an API - of our Anti-Virus solution - to Graylog and therefore ElasticSearch behind that. However, I encouter a very strange mapper-parsing exception (which means that ElasticSearch couldn’t parse the input to the expected datatype). I’ll attach two examples of the same problem right here:

ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [file_attribute_ids] of type [long] in document with id ‘e34ccd28-2a22-11ee-90d7-0242ac100112’. Preview of field’s value: ‘[5, 12]’]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: “[5, 12]”]];

ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [connection_src_port] of type [long] in document with id ‘e1582915-2a22-11ee-90d7-0242ac100112’. Preview of field’s value: ‘63318, 63320’]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: “63318, 63320”]];

So somehow ElasticSearch has a problem parsing these lists to longs. As far as I understood, ElasticSearch also supports multiple values in a field from the ground up, so this list of longs shouldn’t be a problem at all …

Environment:

• OS Information: Linux 5.15.0-73-generic
• Package Version: Graylog 5.1.1+ef1b993 on graylog (Eclipse Adoptium 17.0.7)

What steps have you already taken to try and solve the problem?

• Tried to parse every value in both fields as explicit integers. No success.
• Transformed the datatype of these fields to String. Works until now without any more exceptions.

4. How can the community help?
Please clarify whether this is a bug by ElasticSearch / Graylog or I’m just using these products in an inappropriate way. Thank you very much for your time!

[joe.gross]

Hi @optimaX,

I can explain why it’s not working as data type “long”. According to Elastic, the definition of a long value is:

long A signed 64-bit integer with a minimum value of -263 and a maximum value of 263-1.

I have never heard that Elastic supports multi-values in a long data type.

How did you transform the data type to string? Was it in Graylog or in an Elasticsearch template?

Numeric field types | Elasticsearch Guide [7.17] | Elastic

[optimaX]

Hello Mr. Black,
Seems like I was just confused by this statement in the documentation:
" Arrays:
In Elasticsearch, arrays do not require a dedicated field data type. Any field can contain zero or more values by default, however, all values in the array must be of the same field type." ~ extracted from (Field data types | Elasticsearch Guide [8.8] | Elastic)

However, it seems like the problem consisted of my script forwarding single elements not as arrays including only one element, but forwarding the long value as it is. So Elastic was confused to distinguish between longs and arrays of longs. I only had to make sure to use the array type at all times.

In that script, I transform the events and forward them as logs to Graylog

Thank you very much for your help, I don’t have these mapper errors anymore! Have a great day.

[joe.gross]

Hi @optimaX,

Glad you got it sorted.