We just recently upgraded to a shiny new Graylog 3.0, reindexing our (archaic) Elasticsearch 1.7 cluster to a vigorous young Elasticsearch 6.6.1 cluster, as recommended by the Upgrading Elasticsearch docs.
Everything’s looking truly great, but we have been seeing a lot of Elasticsearch ParseField deprecation warnings since upgrading. After some digging I believe all of these warnings are expected and can be safely ignored, assuming they’re due to the discrepancy between Elasticsearch 6.x and Graylog 3.0 using version 5.6.12 of the Elasticsearch server library.
Do we need to downgrade our Elasticsearch cluster to 5.x, or am I correctly understanding that these deprecation warnings can be ignored?
Details
The warnings are reproducible on a clean Graylog 3.0 server running against a clean Elasticsearch 6.6.1 cluster with a clean MongoDB instance. Depending on the search query, the JSON query generated by the Elasticsearch library uses various query syntaxes deprecated in Elasticsearch 6.x.
For example, the query string (soething else) AND NOT full_message:(x NOT y) leads Graylog (well, the Elasticsearch library) to build the following query:
{
"from": 0,
"size": 150,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "(soething else) AND NOT full_message:(x NOT y)",
"fields": [],
"use_dis_max": true,
"tie_breaker": 0.0,
"default_operator": "or",
"auto_generate_phrase_queries": false,
"max_determinized_states": 10000,
"allow_leading_wildcard": false,
"enable_position_increments": true,
"fuzziness": "AUTO",
"fuzzy_prefix_length": 0,
"fuzzy_max_expansions": 50,
"phrase_slop": 0,
"escape": false,
"split_on_whitespace": true,
"boost": 1.0
}
}
],
"filter": [
{
"bool": {
"must": [
{
"range": {
"timestamp": {
"from": "2019-03-19 15:03:43.132",
"to": "2019-03-19 15:08:43.132",
"include_lower": true,
"include_upper": true,
"boost": 1.0
}
}
}
],
"disable_coord": false,
"adjust_pure_negative": true,
"boost": 1.0
}
}
],
"disable_coord": false,
"adjust_pure_negative": true,
"boost": 1.0
}
},
"sort": [
{
"timestamp": {
"order": "desc"
}
}
]
}
… which in turn triggers the following warnings in the Elasticsearch 6.x cluster:
[2019-03-19T15:08:43,141][WARN ][org.elasticsearch.deprecation.common.ParseField] Deprecated field [use_dis_max] used, replaced by [Set [tie_breaker] to 1 instead]
[2019-03-19T15:08:43,141][WARN ][org.elasticsearch.deprecation.common.ParseField] Deprecated field [auto_generate_phrase_queries] used, replaced by [This setting is ignored, use [type=phrase] instead to make phrase queries out of all text that is within query operators, or use explicitly quoted strings if you need finer-grained control]
[2019-03-19T15:08:43,142][WARN ][org.elasticsearch.deprecation.common.ParseField] Deprecated field [split_on_whitespace] used, replaced by [This setting is ignored, the parser always splits on operator]
[2019-03-19T15:08:43,142][WARN ][org.elasticsearch.deprecation.common.ParseField] Deprecated field [disable_coord] used, replaced by [disable_coord has been removed]
[2019-03-19T15:08:43,142][WARN ][org.elasticsearch.deprecation.common.ParseField] Deprecated field [disable_coord] used, replaced by [disable_coord has been removed]
References
The following PR added compatibility for ES 6 in Graylog 2.5, and seems to indicate some deprecation warnings are still expected:
Thanks in advance for the help, and for how awesome Graylog is! By the way, Graylog handled the upgrade process beautifully!