Snapshot and Restore index

[dazevedo (Diego Azevedo)]

Description of your problem

Good afternoon,

I’m having trouble understanding the backup and restore flow of Elasticsearch indexes and retrieving the snapshot messages directly in the graylog.

I’ve already created elastic retention policies that run every day, and I’m performing the restore tests but I can’t retrieve the messages in Graylog for display.

Operating system information

• Ubuntu Server 20.04

Package versions

• Graylog 4.1.5+01c9198, codename Noir
• MongoDB 4.0.27
• Elasticsearch 7.10.2

[gsmith]

Hello && Welcome

Correct me if I’m wrong, your trying to use Elasticsearch SnapShot & Restore?

If this is correct take a look here about ES SnapShot & Restore documentation.

Snapshot and restore | Elasticsearch Guide [7.10] | Elastic

I execute the Elasticsearch SnapShot & Restore and this post gave me clarity.

Elastic Restore Help Please

@mntbighker I’ve not forgotten about this–it’s just taken longer than I expect because my normal day-to-day duties have kept me pretty busy. I want to ensure I understand what you’re expecting to see. My read is that you’re expecting to see the renamed Graylog indices (both those prefixed w/ graylog- and gl_) just show show up in Graylog–is accurate? If so, there’s additional steps that have to happen for an index to show up in Graylog, and those additional steps are what I was unaware of earlie…

Hope that helps

[dazevedo (Diego Azevedo)]

hello I had already seen this other topic, my biggest doubt is how to make the graylog be able to access the indexes retrieved through the snapshot, if I restore with the graylog_ prefix the graylog understands that this is the default index, sorry if you got confused but not found no documentation on how to access indices restored by elastic in graylog

[gsmith]

Hello,

@dazevedo

no documentation on how to access indices restored by elastic in graylog

I just posted on how Graylog can do this. Take a look at @aaronsachs post again. He explains it very clearly on what you need to do and shows examples.

Elastic Restore Help Please

Any ideas yet @aaronsachs , @konrad or @mpfz0r ?? What we are talking about here is really the ability to combine multiple Graylog instances. This seems to me like a use case that should be documented somewhere. Recovering from a disaster is a process that, to me anyway, looks markedly different from building a fresh instance to migrate your data to. One is planned, and the other is not.

If this is incorrect please show example or steps your doing, that would be helpful

EDIT: I’m sorry I missed this question.

@dazevedo

if I restore with the graylog_ prefix the graylog understands that this is the default index,

Once your done restoring Indices you can manually set your Default Index you want. This is shown here in the red box.

Hope that helps

[dazevedo (Diego Azevedo)]

Hi,

Sorry but restore not easy

follows the prints of the step-by-step executed.

In the end I create a stream to read the restore_graylog_* index messages

however the messages still do not appear.

Does this feature only work correctly in the Enterprise version?

[dazevedo (Diego Azevedo)]

[dazevedo (Diego Azevedo)]

[dazevedo (Diego Azevedo)]

[dazevedo (Diego Azevedo)]

[gsmith]

Hello,

Looks like you have the index in elasticsearch but graylog is not picking it up.

@dazevedo

Does this feature only work correctly in the Enterprise version?

Graylog enterprise version is free, but you need to stay below 5 GB a day.

Small Business

Not sure it will help you restore you index.
There was a note in @aaronsachs post that stated, did you try this?

“could see the index set, but it didn’t seem to indicate that there was anything there until I clicked on it:”

Have you tried rotate/recalculate your index “Restore” manually?

@dazevedo

Sorry but restore not easy

I completely understand, it took me a while to understand how it works and debugging for a couple of days. Once you start to understand it is kind of easy. Just keep working the issue.