Strange indexing error

Graylog Central (peer support)
1

Hello,

I want to ask for help because I ran into a strange error. I ran a Graylog server for several years, which unfortunately once ran out of hard drive. Since the data already collected is absolutely necessary, I took a snapshot of Elasticsearch and then transferred it to another server. The snapshot was successfully restored on the new server, and then I noticed that the old data is present, but no new entries are created, and the following error message is repeated continuously in the Graylog log:

index [graylog_deflector], type [_doc], id [29b82200-a744-11eb-bb87-288023aeccf0], message [ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [graylog_deflector]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]]]

Main server information:

  • Graylog 4.0.6
  • Elasticsearch 7.0.0
  • MongoDB 4.2.1

What I tried, but didn’t help:

  • Restart server.
  • Rotate index.
  • Recalculate index range
  • Set the “index.blocks.read_only_allow_delete” parameter

How could this problem be solved?

2

Hi @toldim

The problem is that your current index has no “graylog_deflector” as an alias.

You can try to fix it manually.

Identify what is the newest graylog index:

curl -s  -XGET "localhost:9200/_cat/indices" | grep 'graylog_' | awk '{print $3}' | sort -t _ -k 2 -rn | head -1

Let’s say your newest index is graylog_20, run this command:

curl -s  -XGET "localhost:9200/graylog_20/_alias?pretty"

You probably will see this output:

{
  "graylog_20" : {
    "aliases" : { }
  }
}

If so, you can follow this documentation to add alias manually.

Hope it helps.

1 Like
3

Hi @reimlima

Thank you for answering ! Unfortunately, this doesn’t seem to be the situation you outlined, because by running the commands I got the following result:

{
  "graylog_79" : {
    "aliases" : {
      "graylog_deflector" : { }
    }
  }
}

Maybe there is something else you can try?

4

Hi,

does this index writeble? Did you tried to manually rotate it via Graylog UI?

5

Hello @reimlima,

Thanks for the help, you gave me a good idea with your answer, which eventually led to the solution. So far, I thought this error message was because graylog could not write any of the indexes. Therefore, I used the following commands:

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/graylog_79/_settings -d '{"index.blocks.read_only": false}'

and

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/graylog_79/_settings -d '{"blocks.read_only": false}'

However, this did not help either, the error message came the same way! That’s when I started thinking about the other part of the error message. As I wrote in the introduction, I migrated the data from one machine to another, which was a newly installed server. It was then that I realized that installing the new instance of graylog in elasticsearch must have created an index for itself. So I looked for the smallest number of graylog indexes, and after removing the graylog_deflector alias from it, the problem was solved right away.

curl -X POST "localhost:9200/_aliases?pretty" -H 'Content-Type: application/json' -d'
{
    "actions" : [
        { "remove" : { "index" : "graylog_0", "alias" : "graylog_deflector" } }
    ]
}

Thanks again for the help!

3 Likes
6

@toldim

Glad to hear that.

Keep going :facepunch:

Have a great one!

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.