[REQ] Question about Error Messages - ElasticsearchException on Graylog 4.1.x

Hi everyone
It has been happening to me lately, with a frequency of 2 or 3 times a month to receive the following error messages

ElasticsearchException[Elasticsearch exception [type=master_not_discovered_exception, reason=FailedToCommitClusterStateException[publication failed]; nested:
ElasticsearchException[publication cancelled before committing: timed out after 30s];]]; nested:
ElasticsearchException[Elasticsearch exception [type=failed_to_commit_cluster_state_exception, reason=publication failed]]; nested:
ElasticsearchException[Elasticsearch exception [type=exception, reason=publication cancelled before committing: timed out after 30s]];

ElasticsearchException[Elasticsearch exception [type=master_not_discovered_exception, reason=NotMasterException[no longer master. source: [put-mapping [graylog_/rMSvNySoSxiTrkTs8SYUZQ]]]]]; nested:
ElasticsearchException[Elasticsearch exception [type=not_master_exception, reason=no longer master. source: [put-mapping [graylog_
/rMSvNySoSxiTrkTs8SYUZQ]]]];

Can you help me decipher the causes?
This is my current configuration

Operating system information

Ubuntu 20.04.3 virtual on Virtualbox version 6.1.28 with Guest Addictions and Extension Pack installed - 64Gb ram, 12 Processor and 600Gb of Virtual Disk Space
Ubuntu 20.04.3 phisical machine with 2 1TB HDD - RAID1 - 256 Gb ram

Package versions

Graylog 4.2.5
MongoDB 4.0.27
Elasticsearch 7.10.2

I use a single node configuration

server.conf

/# Frequency of the Elasticsearch node discovery.
/#
/# Default: 30s
/# elasticsearch_discovery_frequency = 30s

and this is my index rotation

Index prefix:
    graylog
Shards:
    4
Replicas:
    1
Field type refresh interval:
    5 seconds

Index rotation strategy:
    Index Time
Rotation period:
    P1D (1 day, a day)

Index retention strategy:
    Delete
Max number of indices:
    35

Hello @alessio.dapelo

Don’t see your server.conf.

I also don’t see that configuration either.

I believe your issue is that

Does your elasticsearch.yml file look like this?

cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.1.100
http.port: 9200
action.auto_create_index: false
discovery.type: single-node

Just an FYI I had to adjust your post so its easier to read. Please use the markup when posting Config files, Command lines, and/or log files. Thanks :slight_smile:

Could the cause of these issues be the known CPU overuse issue resulting from the Java version following the upgrade of Graylog to versions 4.1.x after the Apache vulnerability mitigation?

@gsmith
My elasticsearch.yml

cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.0.1
http.port: 9200
action.auto_create_index: false

but I can’t find any wording that reports as you reported

discovery.type: single-node

This is my default index setting

Default index set 35 indices, XXXXX documents, XXX GiB default
The Graylog default index set. Graylog will use this index set by default.

Index prefix:    graylog
Shards:    4
Replicas:    1
Field type refresh interval:    5 seconds

Index rotation strategy:    Index Time
Rotation period:    P1D (1 day, a day)

Index retention strategy:    Delete
Max number of indices:    35

It may be a shard setting problem ?

Hello @alessio.dapelo
I’ll explain this a little better.
From the error I see above was what I’m referring to.

The use of discovery.type: single-node Stated here
So this document states:

We recognize that some users need to bind transport to an external interface for testing their usage of the transport client. For this situation, we provide the discovery type single-node (configure it by setting discovery.type to single-node); in this situation, a node will elect itself master and will not join a cluster with any other node.

Unless this is a cluster.

At this point I deduce that I have installed a cluster configured with only one node

Glad you resolved your issue :+1:

I never really said I fixed it

My apologies,

Judging from the title of this post about the “error” message in questioned and you stated

Which I assume because of a misconfiguration, the error went away. Since the error is pointing to not_master_exception, reason=no longer master from this message below.

If this is a single node and as I stated before adding this line in your Elasticsearch YAML file.
discovery.type: single-node

This type of configuration is shown in Elasticsearch documentation.
And I quote:

discovery.type:
(Static) Specifies whether Elasticsearch should form a multiple-node cluster. By default, Elasticsearch discovers other nodes when forming a cluster and allows other nodes to join the cluster later. If discovery.type is set to single-node, Elasticsearch forms a single-node cluster and suppresses the timeout set by cluster.publish.timeout. For more information about when you might use this setting, see Single-node discovery

You can find more here

If this was tried and not working you may need to show more information about your environment.
Hope that helps

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.