Elasticsearch PivotAggregationSearch errors after upgrade to 4.1 ES 7.14

Graylog Central (peer support)
1

Description of your problem

After a more or less successful upgrade to graylog 4.1 and elasticsearch 7.14 we are seeing a lot of the following errors in the graylog server log:
ERROR [PivotAggregationSearch] Aggregation search query returned an error: Elasticsearch exception [type=illegal_argument_exception, reason=maxSize must be >= 0 and < 2147483631; got: 2147483647]

There are no corresponding errors in the elasticsearch log.

Does anyone know what the source of these error is and how to fix them?

Operating system information

  • Ubuntu

Package versions

  • Graylog
    4.1.3+9d79c05

  • MongoDB
    v3.6.23

  • Elasticsearch
    7.14

2

Hello @tgarons,

Unfortunately Elasticsearch 7.14 isnā€™t supported.

https://docs.graylog.org/en/4.0/pages/configuration/elasticsearch.html#elasticsearch-versions

3

We (ā€¦ OK, I :crazy_face:ā€¦) accidentally upgraded elasticsearch to 7.14 and I am seeing those errors in my Graylog log as well. I also see that my index registers as having been created 52 years ago (punch cards, I am sure!) From what I can tell so far this results in widgets on dashboards failing to load properly on occasion - usually a single refresh will fix that. Had I been a bit more astute I would have caught that we shouldnā€™t upgrade Elastic when doing a general Ubuntu upgrade ā€¦ and I would have put a hold on Elastic to 7.10.

I was considering adding a 7.10 version to the cluster, then dropping/rebuilding the 7.14 machines but I am not sure it is possible - Elastic cluster may choke on versioning or data may get lost/corrupted. Stillā€¦it would be a good exercise to go though thoughā€¦ :thinking:

2 Likes
4

At the very least a package hold to prevent future accidental upgrades is warranted, which is what I did after the upgrade to 7.12.

1 Like
5

Let me know how adding a 7.10 node works out for you. As I totally hosed three months worth of data when I upgraded from 5.6->6.8->7.14 I may just start from scratch, but if there was a smoother way to get a supported release that would be great. As it is things seem to be working for the most part aside from those errorsā€”the stuff I care aboutā€”streams, alerts, plugin configurations all seem functional. If 7.14 is on the road map Iā€™m willing to live with the errors for a while.
Tom

1 Like
6

Hello,

I had this happen to myself a couple times and what I started doing after a install was pinning by repository name.

https://itectec.com/ubuntu/ubuntu-pinning-package-using-own-repository-and-apt-get/#:~:text=Solution%202%20(using%20Pinning)

7

Same problem here, everything was working fine until I add a third graylog node to complete my cluster,

now I get a lot of ā€œERROR [PivotAggregationSearch]ā€ in my logs resulting to a non working aggregation so now all my alerts with aggregation results wonā€™t work anymoreā€¦
Is there a way to fix this please?

Graylog version: 4.0.11
ES version: 7.14

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.