Restore Elasticsearch Cluster data to new Graylog server

Graylog Central (peer support)
1

1. Describe your incident:
Hello All,

I have an issue with Graylog that I am unable to solve. I am currently testing out backup and restore. I am doing this using Elasticsearch snapshots. The goal is to copy the data from the existing Graylog LMS to the new LMS.

So this is the setup. We have a Graylog 4.2.13 that is collecting logs which are being stored in an Elasticsearch cluster with 2 nodes in it. I have taken a snapshot of this cluster. I have a new Graylog server running version 5.0.0 which is being supported by an Elasticsearch cluster with 3 nodes. I was able to restore the snapshot to the new cluster and I am also able to verify that all the indices have been restored. However when I used the Graylog5 GUI to look at the index sets, at first I was able to see only the default index set had any data in it.

So I looked around and found this thread Elastic Restore Help Please - #16 by aaronsachs. I followed the suggestion posted there and created the index sets on the Graylog5 server exactly as they existed on the Graylog 4.3.3 server. As soon as I created the index set, I was able to see that it was recognizing the indices and it populated with the right number of indices. Also the size of the overall index set is also correct. However when I try to search the logs, I am able to see logs only from the default index set and not from the other index sets. I have 2 more index sets that I need to be able to read.

I stopped the graylog service before restoring the snapshot to the new cluster. I also tried it without stopping the graylog service. I also created the index sets on the new server as they exist on the old server and then restored as well. All of the above result in the same behavior, I am able to read the default index set but not the other 2 index sets.

I wanted to ask the community if anyone has seen this before and if I am doing something incorreclty. Also if there is a solution I will be grateful for it.

Everything is running on RHEL 7.9 and elasticsearch version is 7.10.2 on both the new and the old clusters.

Thank you

2

The first thing that comes to mind that I didn’t see you mention is to recalculate the index ranges on the new server once you have brought the data over - you may even try rotating the active write index… (try that second).

image

3

Thank you for responding.

I did recalculate the index ranges on the new server and it calculates the range same as on the old server. Just missed mentioning it.

I tried rotating the index just now, it created a new index but there are no incoming messages currently so it is just an empty index for now.

No change, still unable to read data from the 100 or so older indexes.

4

I am planning an upgrade for some time when I feel like I have time to dedicate to it… my plan is to look at reindexing from the old server to the new server - something starting with this:

Haven’t looked into the details but thought it might be helpful to think in that direction too…

5

Thank you, I will take a look at that article and see if there is something in there that may fix my issue. If it does, I will post an update here.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.