Graylog backups

[dbcrvl]

Hi,

I have been handed over a Graylog server which has no backups set up and would like to do a daily/weekly backup to another server.

I am mainly interested in the configurations, dashboards, pipelines/rules, extractors… rather than the data itself, but having a full backup would also help. Basically, what I need to make sure is in case of failure I can just build a new server and restore the old server’s data on to it without having to reconfigure anything or create dashboards, etc. again.

What’s the process for doing this?

Thanks,

[macko003]

Hi

A graylog system work with 3 components.
You should backup each one.
Generally - backup each config file.
Mongodb - contains the graylog settings., you can backup with mongodump
Elasticsearch - contains the logs - you can check elastic snapshots or elasticdump
Graylog - You need the config file only.

I use this script

#!/bin/bash
HOST=`hostname`
/usr/bin/rm -f /backup/graylog_components_config.$HOST.tar.4
/usr/bin/mv -f /backup/graylog_components_config.$HOST.tar.3 /backup/graylog_components_config.$HOST.tar.4
/usr/bin/mv -f /backup/graylog_components_config.$HOST.tar.2 /backup/graylog_components_config.$HOST.tar.3
/usr/bin/mv -f /backup/graylog_components_config.$HOST.tar.1 /backup/graylog_components_config.$HOST.tar.2
/usr/bin/mv -f /backup/graylog_components_config.$HOST.tar /backup/graylog_components_config.$HOST.tar.1
/usr/bin/tar cvf /backup/graylog_components_config.$HOST.tar /etc/mongodb-keyfile /etc/mongod.conf /etc/elasticsearch/elasticsearch.yml /etc/graylog/server/server.conf /etc/graylog/server/node-id
/usr/bin/rm -fr /backup/$HOST.mongodump.4
/usr/bin/mv -f /backup/$HOST.mongodump.3 /backup/$HOST.mongodump.4
/usr/bin/mv -f /backup/$HOST.mongodump.2 /backup/$HOST.mongodump.3
/usr/bin/mv -f /backup/$HOST.mongodump.1 /backup/$HOST.mongodump.2
/usr/bin/mv -f /backup/$HOST.mongodump /backup/$HOST.mongodump.1
/usr/bin/mongodump --host $HOST:27017 -u USER -p PASS -d DATABASE -o /backup/$HOST.mongodump

And a script to make snapshot based on the official solution
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

[dbcrvl]

Thanks macko003, could not look at it on my server but will do once I have a chance and let you know how it went.

[Totally_Not_A_Robot (Tess)]

I imagine that with your setup (13TB a month) you’re not going for anything special in the Elastic backups, are you? Our logs are much, much smaller so we’re basically grabbing snapshots of our VMs from time to time, relying upon Elastic’s replication for actual higher availability of the data.

[macko003]

Nothing special in my elastic backup script.
I got it from google:) but I have 4 index sets, so I’m not proud about my script

I used something like that as template
https://www.karelbemelmans.com/2015/03/elasticsearch-backup-script-with-snapshot-rotation/