Indexer error: {“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}
2. Describe your environment:
CentOS 7
Package Version:
MongoDB 3.2.22 + Elasticsearch 5.6.16 + GrayLog 3.0.2
(will be updated to MongoDB 3.6, Elasticsearch 6.8, GrayLog 3.3.16 soon)
Service logs, configurations, and environment variables:
Indexer error: {“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}
3. What steps have you already taken to try and solve the problem?
Is [BIG_INTEGER] something you added to remove the number there is is that the actual error? If it is the actual error, it looks like you are setting the field type to be BIG_INTEGER which doesn’t exist… need more context about where this is happening, maybe an example message and some examples of how you are working on parsing that message such as extractors or pipeline rules…?
Hello,
The error is displayed in the indexer error log list. Unfortunately there is no context, I have absolutely no clue which message does trigger this error.
We have not used BIG_INTEGER in any config.
Where could I get more information about this error?
There is some information on it out on the interwebs here and here
in short, one possibility is you have a number defined as a float when it should be long. You may need to do a custom index to ensure that elastic stores the correct field type.
I have inherited this installation.
We only have configured some inputs (regex, grok, lookup table, json) .
I am not aware of any configuration containing explicitly defined data types.
Hello,
I can not find any location where we have configured a data type BIG_INTEGER.
Any hint where in the Graylog configuration I could possible find something?
Looks like BIG_INTEGER is just part of the error message… It may be the field is defined as a float when it should be defined as a long (larger values).
Hello,
Yes, that might be the case, but I did not define any data types for fields in Graylog (I just inherited the installation). To my knowledge there was noting manually configured in Elasticsearch.
So my problem is where in Graylog are data types part of the configuration.
I did not find anything tn the input/stream configuration. It might be an implicit thing.
There is no configuration in Graylog. Elasticsearch takes a guess a what the type is each time the index rotates. As the data comes in to the new index the field is reevaluated. You can find out what type each field is with( you can use wildcards with index names and grep out for a float) - fingers crossed that is the issue since it is easy to fix.
curl -X GET "<elastic_server_name>:9200/<index name>/_mapping?pretty"
2022-02-15T14:07:29.019+01:00 WARN [Messages] Failed to index message: index=<graylog_12022> id=<3a383711-8e60-11ec-95e2-506b8dbcf4f4> error=<{“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}>
2022-02-15T14:07:29.023+01:00 ERROR [Messages] Failed to index [1] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2022-02-15T14:07:37.308+01:00 INFO [AbstractRotationStrategy] Deflector index (index set <graylog_12022>) should be rotated, Pointing deflector to neindex now!
2022-02-15T14:07:37.309+01:00 INFO [MongoIndexSet] Cycling from <graylog_12022> to <graylog_12023>.
2022-02-15T14:07:37.309+01:00 INFO [MongoIndexSet] Creating target index <graylog_12023>.
2022-02-15T14:07:37.316+01:00 INFO [Indices] Successfully created index template graylog-internal
2022-02-15T14:07:37.341+01:00 INFO [MongoIndexSet] Waiting for allocation of index <graylog_12023>.
2022-02-15T14:07:37.342+01:00 INFO [MongoIndexSet] Index <graylog_12023> has been successfully allocated.
In /var/log/elasticsearch/graylog.log:
[2022-02-28T04:37:38,863][DEBUG][o.e.a.b.TransportShardBulkAction] [Yf73DoX] [graylog_12616][0] failed to execute bulk item (index) BulkShardRequest [[graylog_12616][0]] co
ntaining [374] requests
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:176) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.6.16.jar:5.6.16]
…
Caused by: java.lang.IllegalStateException: No matching token for number_type [BIG_INTEGER]
at org.elasticsearch.common.xcontent.json.JsonXContentParser.convertNumberType(JsonXContentParser.java:213) ~[elasticsearch-5.6.16.jar:5.6.16]
