Excessive Indexer Failures - Opensearch - Of like kind

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:

Excessive Indexer Errors like:

OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=failed to parse field [ListBaseType] of type [long] in document with id 'c0b9fbc0-c8c5-11ed-895a-00163ef2bcdd'. Preview of field's value: 'GenericList']]; nested: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=For input string: "GenericList"]];

2. Describe your environment:
OpenSearch 2.0.1
Server OS: Ubuntu 22.04 in LXC guest; LXC host: Debian Bookworm
Graylog: 5.0.3+a82acb2 (open/community edition)

3. What steps have you already taken to try and solve the problem?

The usual Google-foo

Thank you!

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

In the INdex this message is going to the field ListBaseType is slated as type long but the data from the message you are trying to inject into it is of type string with the word GenericList

When indexes rotate, they refresh field type based on the first message that comes in on a field. SO when that index rotated, the next message had something 222 for the ListBaseType field and OpenSource decided the field will now be a long.

Short/quick fix is to rotate the index and hopefully the next message coming in is the correct string type. Long term you can set a custom mapping to the field to force it to be the correct type (in case all-numerics occasionally happen.

I wrote up how to do it in an older version of ElasticSearch here - that may give you some tips if you want to do a custom mapping - likely slightly different in OpenSearch now.

3

Sweet fancy Moses … I have a lot of reading to do before I attempt that (this is production). Thank you!