Failures During ES6 Reindex for Graylog

Graylog Central (peer support)
1

I have been working my way through the procedures outlined in Elasticsearch Reindexing Notes — Graylog 4.1.0 documentation to reindex all of my original ES5 Graylog indices under ES6 after upgrade to 6.8; been running reindex operations for days now.

I just ran into some failures with one of the original indices and I’m curious as to the best way to deal with these failures such that it doesn’t negatively affect Graylog, as well as how to avoid the situation in the future. Any help would be greatly appreciated.

"failures": [
    {
        "cause": {
            "caused_by": {
                "reason": "Failed to parse value [2020-05-12T00:56:03.400Z] as only [true] or [false] are allowed.",
                "type": "illegal_argument_exception"
            },
            "reason": "failed to parse field [regex_matches] of type [boolean] in document with id '790304a4-93eb-11ea-b76a-ac1f6b0ac5b8'",
            "type": "mapper_parsing_exception"
        },
        "id": "790304a4-93eb-11ea-b76a-ac1f6b0ac5b8",
        "index": "graylog_0_reindex",
        "status": 400,
        "type": "message"
    },
    {
        "cause": {
            "caused_by": {
                "reason": "Failed to parse value [2020-05-12T00:56:03.399Z] as only [true] or [false] are allowed.",
                "type": "illegal_argument_exception"
            },
            "reason": "failed to parse field [regex_matches] of type [boolean] in document with id '790304a2-93eb-11ea-b76a-ac1f6b0ac5b8'",
            "type": "mapper_parsing_exception"
        },
        "id": "790304a2-93eb-11ea-b76a-ac1f6b0ac5b8",
        "index": "graylog_0_reindex",
        "status": 400,
        "type": "message"
    },
    {
        "cause": {
            "caused_by": {
                "reason": "Failed to parse value [2020-05-12T00:56:03.398Z] as only [true] or [false] are allowed.",
                "type": "illegal_argument_exception"
            },
            "reason": "failed to parse field [regex_matches] of type [boolean] in document with id '790304a0-93eb-11ea-b76a-ac1f6b0ac5b8'",
            "type": "mapper_parsing_exception"
        },
        "id": "790304a0-93eb-11ea-b76a-ac1f6b0ac5b8",
        "index": "graylog_0_reindex",
        "status": 400,
        "type": "message"
    }
]
2

Your mapping is not correct in your index set. You are attempting to put in a date into your index where currently only boolean values are accepted. Try mapping your index/index set correctly.

3

@edwardmarrufo

Thanks, I gathered that from the error message. However, I did not set the mapping for this or any other index; Graylog did. Given the name of the index, I’m assuming this was the first one Graylog created when I deployed it last year. I’ve not had this issue with reindex operations for any other Graylog indices (graylog_*). So how do I correct this mapping in a way that Graylog supports?

4

Hello

Have you seen this?

5

Yes, I know how to locate and read the Elasticsearch documentation. :face_with_raised_eyebrow:

I was looking for something more prescriptive and contextual to Graylog. As I mentioned previously, all other Graylog indices (graylog_7, graylog_8, graylog_9, etc.) haven’t had this issue; only graylog_0.

The reindex task in which the failures occur actually processed over 700K documents before it reached this failure. Does this indicate some sort of corruption in that index?

response": {
        "batches": 703,
        "created": 702997,

My deployment started as Graylog 3.x with ES5 last year, but I can’t recall the exact minor version of Graylog 3.x. I’m on 4.1 now, with ES6.

This reminds me of the Graylog reindex documentation, which provides a command example for reviewing the index mappings, but no guidance on what they should be based on Graylog templates and such.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.