Graylog Opensearch cluster is yellow

Hi,

I have my graylog server with 1 opensearch and it has been working fine for the last 3 months.

Everything is working, but in indices section of the graylog server, it is showing that “the Elasticsearch cluster is yellow”

How can I solve?

Hey @david.oliveira

It states Unassinged shards thats probably why its in yellow.

This will show your primary and replica shards ( i.e., p=primary , r = replica)

curl -XGET http://localhost:9200/_cat/shards

if you noticed that unassigned shards has a “r” , meaning your replicas are unassinged. To find out why execute this cURL command. it should explain it in more detail.

curl -XGET http://localhost:9200/_cluster/allocation/explain?pretty

Normally replicas are assigned to a different node if this is the case.

Hi @gsmith

Thanks for the reply, I ran this command and it gave me this result.
curl -XGET http://localhost:9200/_cat/shards

index.asp          0 p STARTED           0   208b 10.x.x.33 srv
index.asp          0 r UNASSIGNED
graylog_0          2 p STARTED    36642595 22.8gb 10.x.x.33 srv
graylog_0          3 p STARTED    36637113 22.8gb 10.x.x.33 srv
graylog_0          1 p STARTED    36642547 22.8gb 10.x.x.33 srv
graylog_0          0 p STARTED    36648665 22.8gb 10.x.x.33 srv
index.php          0 p STARTED           0   208b 10.x.x.33 srv
index.php          0 r UNASSIGNED
gl-events_0        2 p STARTED           0   208b 10.x.x.33 srv
gl-events_0        3 p STARTED           0   208b 10.x.x.33 srv
gl-events_0        1 p STARTED           0   208b 10.x.x.33 srv
gl-events_0        0 p STARTED           0   208b 10.x.x.33 srv
gl-system-events_1 2 p STARTED           0   208b 10.x.x.33 srv
gl-system-events_1 3 p STARTED           0   208b 10.x.x.33 srv
gl-system-events_1 1 p STARTED           0   208b 10.x.x.33 srv
gl-system-events_1 0 p STARTED           1 10.8kb 10.x.x.33 srv
index.py           0 p STARTED           0   208b 10.x.x.33 srv
index.py           0 r UNASSIGNED
index.pl           0 p STARTED           0   208b 10.x.x.33 srv
index.pl           0 r UNASSIGNED
index.action       0 p STARTED           0   208b 10.x.x.33 srv
index.action       0 r UNASSIGNED
index.do           0 p STARTED           0   208b 10.x.x.33 srv
index.do           0 r UNASSIGNED
index.htm          0 p STARTED           0   208b 10.x.x.33 srv
index.htm          0 r UNASSIGNED
index.cfm          0 p STARTED           0   208b 10.x.x.33 srv
index.cfm          0 r UNASSIGNED
index.cgi          0 p STARTED           0   208b 10.x.x.33 srv
index.cgi          0 r UNASSIGNED
index.jsp          0 p STARTED           0   208b 10.x.x.33 srv
index.jsp          0 r UNASSIGNED
graylog_2          2 p STARTED     2660071  1.8gb 10.x.x.33 srv
graylog_2          3 p STARTED     2657873  1.8gb 10.x.x.33 srv
graylog_2          1 p STARTED     2659034  1.8gb 10.x.x.33 srv
graylog_2          0 p STARTED     2665237  1.8gb 10.x.x.33 srv
index.aspx         0 p STARTED           0   208b 10.x.x.33 srv
index.aspx         0 r UNASSIGNED
graylog_1          2 p STARTED    19843739 13.1gb 10.x.x.33 srv
graylog_1          3 p STARTED    19840155 13.1gb 10.x.x.33 srv
graylog_1          1 p STARTED    19845908 13.1gb 10.x.x.33 srv
graylog_1          0 p STARTED    19837151 13.1gb 10.x.x.33 srv
gl-system-events_0 2 p STARTED           1   10kb 10.x.x.33 srv
gl-system-events_0 3 p STARTED           1 10.8kb 10.x.x.33 srv
gl-system-events_0 1 p STARTED           0   208b 10.x.x.33 srv
gl-system-events_0 0 p STARTED           0   208b 10.x.x.33 srv
index.html         0 p STARTED           0   208b 10.x.x.33 srv
index.html         0 r UNASSIGNED

I have only one Opensearch node and it’s all together on the same server

And here is the result of this command
curl -XGET http://localhost:9200/_cluster/allocation/explain?pretty

{
  "index" : "index.asp",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2023-06-02T23:24:30.243Z",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "xXXxxXxxXX-xXJEGajgg",
      "node_name" : "srv",
      "transport_address" : "10.x.x.33:9300",
      "node_attributes" : {
        "shard_indexing_pressure_enabled" : "true"
      },
      "node_decision" : "no",
      "weight_ranking" : 1,
      "deciders" : [
        {
          "decider" : "same_shard",
          "decision" : "NO",
          "explanation" : "a copy of this shard is already allocated to this node [[index.asp][0], node[XxxxxXxBie-xXJEGajgg], [P], s[STARTED], a[id=xxXXxxXXx1tuglflNkQ]]"
        }
      ]
    }
  ]
}

Do you think this is normal?

If you only have 1 opensearch node, this is normal.

When you create an index, it may create replica shards as well. Replica shards are copies of the data. This is useful for fault tolerance when you have 2 or more opensearch nodes and you can have multiple copies of the data in case a node fails.

If you only have a single opensearch node, the replica shards do not have anywhere to be assigned since they can’t live on the same node as the primary shards. See About OpenSearch - OpenSearch documentation

To resolve this, you can update the index to set the number of replicas to 0:

curl -X PUT "http://127.0.0.1:9200/<index name>/_settings" -H 'Content-Type: application/json' -d '{"index":{"number_of_replicas":0}}'

replace <index name> with the index you want to update. You can also use * as a wildcard so if you did index.* it would match all indices starting with index..

Hope that helps.

2 Likes

In the index, the replica is already at 0, but it is normal to continue in yellow

Hey @david.oliveira

Have you execute a index set rotation manually once replicas were set to “0”?
Just to confirm these index set are all set to “0”

  • index.asp
  • index.php
  • index.py
  • index.pl
  • index.action
  • index.do
  • index.htm
  • index.cfm
  • index.cgi
  • index.jsp
  • index.aspx
  • index.html

If so, aand you still have these unassigned replicas maybe check out this site.

I had to use that when working with Opensearch while back. Remember to back things up :smile:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.