Unsupported parameters in Custom Mapping

Hello Community,
I’m running Graylog 3.0 and Elasticsearch 6.8 and I have to create a custom mapping to prevent any issue/warning while logs are written on ES.

I’ve created the following custom mapping:

{
“template”: “graylog_*”,
“mappings”: {
“message”: {
“properties”: {
“file”: {
“type”: “text”
},
“Type”: {
“type”: “text”
},
“X-B3-SpanId”: {
“type”: “text”
},
“full_message”: {
“type”: “keyword”,
“ignore_above”: 4096
},
“activityName”: {
“type”: “text”
},
“className”: {
“type”: “text”
},
“level”: {
“type”: “long”
},
“cab-ttl”: {
“type”: “long”,
“null_value”: -99,
“ignore_malformed”: true
},

        }
    }
}

}

Once successfully loaded in Elasticsearch, the indexes don’t rotate any longer and logs stay in the buffer until I delete the mapping.

The error I get is:

WARN [Indices] Couldn’t create index graylog_4. Error: {“root_cause”:[{“type”:“mapper_parsing_exception”,“reason”:“Mapping definition for [full_message] has unsupported parameters: [fielddata : false] [analyzer : standard]”}],“type”:“mapper_parsing_exception”,“reason”:“Failed to parse mapping [message]: Mapping definition for [full_message] has unsupported parameters: [fielddata : false] [analyzer : standard]”,“caused_by”:{“type”:“mapper_parsing_exception”,“reason”:“Mapping definition for [full_message] has unsupported parameters: [fielddata : false] [analyzer : standard]”}}
[…]
ERROR [IndexRotationThread] Couldn’t point deflector to a new index

Fielddata and analyzer are standard parameters for Elasticsearch (and I haven’t changed them anywhere)

I don’t understand what I’m doing wrong and I haven’t found any suitable solution or hint in internet.

Do you have any suggestion?

Thanks!

he @gabbar
you might want to look into this: https://www.elastic.co/blog/strings-are-dead-long-live-strings

The last entry should be your solution:

{
  "foo": {
    "type" "keyword",
    "index": true
  }
}

Hi @jan,
unfortunately it doesn’t match my problem. I’m already using the keyword datatype, and with or without “index” the situation doesn’t change, moreover I have to limit the field at 4096 characters.

Using the solution you’ve suggested, the output is

2020-01-27T14:28:14.897+01:00 WARN  [Indices] Couldn't create index graylog_16920. Error: {"root_cause":[{"type":"mapper_parsing_exception","reason":"Mapping definition for [full_message] has unsupported parameters:  [fielddata : false] [analyzer : standard]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [message]: Mapping definition for [full_message] has unsupported parameters:  [fielddata : false] [analyzer : standard]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [full_message] has unsupported parameters:  [fielddata : false] [analyzer : standard]"}}
2020-01-27T14:28:14.897+01:00 ERROR [IndexRotationThread] Couldn't point deflector to a new index

if I try to to force the “text” datatype instead… the server says that I should use a keyword (which doesn’t work at all)

2020-01-27T14:20:45.079+01:00 ERROR [CreateNewSingleIndexRangeJob] Exception during index range calculation for index graylog_16918
org.graylog2.indexer.ElasticsearchException: Couldn't build index range of index graylog_16918
Fielddata is disabled on text fields by default. Set fielddata=true on [streams] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
            at org.graylog2.indexer.cluster.jest.JestUtils.specificException(JestUtils.java:98) ~[graylog.jar:?]
            at org.graylog2.indexer.cluster.jest.JestUtils.execute(JestUtils.java:57) ~[graylog.jar:?]
            at org.graylog2.indexer.cluster.jest.JestUtils.execute(JestUtils.java:62) ~[graylog.jar:?]
            at org.graylog2.indexer.indices.Indices.indexRangeStatsOfIndex(Indices.java:741) ~[graylog.jar:?]

Of course I’m still haven’t written the unsupported parameters anywhere, should I maybe force them?

Kind regards

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.