Issues after upgrading to 3.0.0

Graylog Central (peer support)
1

Hello,

I had previously been on version 2.6. I have gone though the steps of setting up new elasticsearch 6.x nodes and doing the remote reindex from my 5.x elasticsearch nodes. Cluster health shows that the index(s) are good. However when I start I see the following in my logs:

Fielddata is disabled on text fields by default. Set fielddata=true on [timestamp] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.Fielddata is disabled on text fields by default. Set fielddata=true on [timestamp] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead. (ElasticsearchException)

I have tried to follow:

http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings

To create a custom field mapping but that yields this error:

{

“error” : {
“root_cause” : [
{
“type” : “mapper_parsing_exception”,
“reason” : “No handler for type [string] declared on field [http_method]”
}
],
“type” : “mapper_parsing_exception”,
“reason” : “Failed to parse mapping [message]: No handler for type [string] declared on field [http_method]”,
“caused_by” : {
“type” : “mapper_parsing_exception”,
“reason” : “No handler for type [string] declared on field [http_method]”
}
},
“status” : 400
}

I am at a lose for how to fix this.

2

We’re missing the one essential element here, your custom field mapping :wink: Could you paste it here so we can see what you tried to do?

3

How do I supply this information to you followin this document:

http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings

When I try to do the mapping suggested it complains about the string type specified:

“http_method” : {
“type” : “string”,
“index” : “not_analyzed”
},

Is this syntax not correct?

I think this is what you are asking for:

nclark@nclark:~$ curl -X GET 'http://vroomfondel:9200/template/graylog-internal?pretty’
{
“graylog-internal” : {
“order” : -1,
“index_patterns” : [
"graylog"
],
“settings” : {
“index” : {
“analysis” : {
“analyzer” : {
“analyzer_keyword” : {
“filter” : “lowercase”,
“tokenizer” : “keyword”
}
}
}
}
},
“mappings” : {
“message” : {
“properties” : {
“message” : {
“type” : “text”,
“analyzer” : “standard”,
“fielddata” : false
},
“full_message” : {
“type” : “text”,
“analyzer” : “standard”,
“fielddata” : false
},
“timestamp” : {
“type” : “date”,
“format” : “yyyy-MM-dd HH:mm:ss.SSS”
},
“source” : {
“type” : “text”,
“analyzer” : “analyzer_keyword”,
“fielddata” : true
},
“streams” : {
“type” : “keyword”
}
},
“dynamic_templates” : [
{
“internal_fields” : {
“match” : "gl2_",
“mapping” : {
“type” : “keyword”
}
}
},
{
“store_generic” : {
“match_mapping_type” : “string”,
“mapping” : {
“type” : “keyword”
}
}
}
],
“_source” : {
“enabled” : true
}
}
},
“aliases” : { }
}
}

4

if would you would format your posting like mentioned in the FAQ: https://community.graylog.org/faq#format-markdown it is more likely that someone is able to help you in his spare time …

1 Like
5

Alternatively throw it on pastebin and paste the link here :slight_smile: Easier to read when something is properly formatted :smiley:

7

All I am looking for is the correct way to create the graylog-custom-mapping.json.

I tried as it says in http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings

nclark@nclark:~$ curl -X GET 'http://vroomfondel:9200/template/graylog-internal?pretty’

It outputs the following:

“error” : {
“root_cause” : [
{
“type” : “mapper_parsing_exception”,
“reason” : “No handler for type [string] declared on field [http_method]”
}
],
“type” : “mapper_parsing_exception”,
“reason” : “Failed to parse mapping [message]: No handler for type [string] declared on field [http_method]”,
“caused_by” : {
“type” : “mapper_parsing_exception”,
“reason” : “No handler for type [string] declared on field [http_method]”
}
},
“status” : 400
}

I do not think there is a field type of string anymore in elasticsearch 6.x. I am not sure how to update this to work. I got it all to mostly work by removing indexes I moved in from 2.x elasticsearch. Though my AWS plugin will not work and spits out the same error message about field index.

8

Well I ended up just removing all the index(s) I transferred in from the older elasticsearch and those messages have disappeared.

I guess we can just close this post

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.