Indexer error: mapper_parsing_exception No matching token for number_type [BIG_INTEGER]

1. Describe your incident:

Indexer error: {“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}

2. Describe your environment:

CentOS 7

  • Package Version:

MongoDB 3.2.22 + Elasticsearch 5.6.16 + GrayLog 3.0.2
(will be updated to MongoDB 3.6, Elasticsearch 6.8, GrayLog 3.3.16 soon)

  • Service logs, configurations, and environment variables:

Indexer error: {“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}

3. What steps have you already taken to try and solve the problem?

Automatic house keeping of GrayLog.

4. How can the community help?

How can I find the problem causing this error?

Rainer

Is [BIG_INTEGER] something you added to remove the number there is is that the actual error? If it is the actual error, it looks like you are setting the field type to be BIG_INTEGER which doesn’t exist… need more context about where this is happening, maybe an example message and some examples of how you are working on parsing that message such as extractors or pipeline rules…?

Hello,
The error is displayed in the indexer error log list. Unfortunately there is no context, I have absolutely no clue which message does trigger this error.

We have not used BIG_INTEGER in any config.

Where could I get more information about this error?

Bye
Rainer

There is some information on it out on the interwebs here and here

in short, one possibility is you have a number defined as a float when it should be long. You may need to do a custom index to ensure that elastic stores the correct field type.

1 Like

Hello,
Thanks, I will check the provided links.

I have inherited this installation.
We only have configured some inputs (regex, grok, lookup table, json) .
I am not aware of any configuration containing explicitly defined data types.

Bye
Rainer

Hello,
I forgot one thine: We did not manually configure anything on Elasticsearch, everything if configured via the Graylog GUI.

Bye
Rainer

Hello,
I can not find any location where we have configured a data type BIG_INTEGER.
Any hint where in the Graylog configuration I could possible find something?

Bye
Rainer

Looks like BIG_INTEGER is just part of the error message… It may be the field is defined as a float when it should be defined as a long (larger values).

Hello,
Yes, that might be the case, but I did not define any data types for fields in Graylog (I just inherited the installation). To my knowledge there was noting manually configured in Elasticsearch.

So my problem is where in Graylog are data types part of the configuration.
I did not find anything tn the input/stream configuration. It might be an implicit thing.

Bye
Rainer

There is no configuration in Graylog. Elasticsearch takes a guess a what the type is each time the index rotates. As the data comes in to the new index the field is reevaluated. You can find out what type each field is with( you can use wildcards with index names and grep out for a float) - fingers crossed that is the issue since it is easy to fix. :slight_smile:

curl -X GET "<elastic_server_name>:9200/<index name>/_mapping?pretty"

1 Like

Hello,
Thanks very much, I’ll have a look. So far I did not see any float or int.
I see type: keyword, long, date, text

Bye
Rainer

Is there anything else in your Graylog log that is helpful/relevant? Maybe this happens on a particular query?

tail -f /var/log/graylog-server/server.log

Hello,
I can see the following:

2022-02-15T14:07:29.019+01:00 WARN [Messages] Failed to index message: index=<graylog_12022> id=<3a383711-8e60-11ec-95e2-506b8dbcf4f4> error=<{“type”:“mapper_parsing_exception”,“reason”:“failed to parse”,“caused_by”:{“type”:“illegal_state_exception”,“reason”:“No matching token for number_type [BIG_INTEGER]”}}>
2022-02-15T14:07:29.023+01:00 ERROR [Messages] Failed to index [1] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2022-02-15T14:07:37.308+01:00 INFO [AbstractRotationStrategy] Deflector index (index set <graylog_12022>) should be rotated, Pointing deflector to neindex now!
2022-02-15T14:07:37.309+01:00 INFO [MongoIndexSet] Cycling from <graylog_12022> to <graylog_12023>.
2022-02-15T14:07:37.309+01:00 INFO [MongoIndexSet] Creating target index <graylog_12023>.
2022-02-15T14:07:37.316+01:00 INFO [Indices] Successfully created index template graylog-internal
2022-02-15T14:07:37.341+01:00 INFO [MongoIndexSet] Waiting for allocation of index <graylog_12023>.
2022-02-15T14:07:37.342+01:00 INFO [MongoIndexSet] Index <graylog_12023> has been successfully allocated.

In /var/log/elasticsearch/graylog.log:

[2022-02-28T04:37:38,863][DEBUG][o.e.a.b.TransportShardBulkAction] [Yf73DoX] [graylog_12616][0] failed to execute bulk item (index) BulkShardRequest [[graylog_12616][0]] co
ntaining [374] requests
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:176) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.6.16.jar:5.6.16]

Caused by: java.lang.IllegalStateException: No matching token for number_type [BIG_INTEGER]
at org.elasticsearch.common.xcontent.json.JsonXContentParser.convertNumberType(JsonXContentParser.java:213) ~[elasticsearch-5.6.16.jar:5.6.16]

No user queries were executed at this time.

Bye
Rainer

Here is one of the links I was looking at:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.