Hi
I am using a kubernetes install of graylog and am trying to put together a disaster recovery plan.
I have scripted the mongodb database and saved the graylog file specified in the doc. I’m not worried about elasticsearch for now.
However; How do i use them??
I just need to recover the config to a new install … I tried just restoring the mongo backup (with --drop ) but this leaves a problem with the index names. Then I tried to restore collection by collection which seems better until i got to streams… I can manage all the streams except the default all index … looks like its pointing at a non existent index and I cannot manage the settings (impossible in the gui)?? can any one help??
Even better would be some info in the documentation explaining how to perform a disaster recovery to a new install using the file mentioned in the docs!! for once reading the docs is not helping…
Peter
info
index_sets collection
rs0:PRIMARY> db.index_sets.find( {} )
{ “_id” : ObjectId(“5f0cdd9719bb63000c1925a4”), “title” : “Default index set”, “description” : “The Graylog default index set”, “index_prefix” : “graylog”, “shards” : 4, “replicas” : 0, “rotation_strategy_class” : “org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategy”, “rotation_strategy” : { “type” : “org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig”, “max_docs_per_index” : 20000000 }, “retention_strategy_class” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy”, “retention_strategy” : { “type” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig”, “max_number_of_indices” : 20 }, “creation_date” : ISODate(“2020-07-13T22:17:59.554Z”), “index_analyzer” : “standard”, “index_template_name” : “graylog-internal”, “index_template_type” : null, “index_optimization_max_num_segments” : 1, “index_optimization_disabled” : false, “field_type_refresh_interval” : NumberLong(5000), “writable” : true }
{ “_id” : ObjectId(“5f0cdda419bb63000c192639”), “title” : “Graylog Events”, “description” : “Stores Graylog events.”, “index_prefix” : “gl-events”, “shards” : 4, “replicas” : 0, “rotation_strategy_class” : “org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy”, “rotation_strategy” : { “type” : “org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig”, “rotation_period” : “P1M” }, “retention_strategy_class” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy”, “retention_strategy” : { “type” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig”, “max_number_of_indices” : 12 }, “creation_date” : ISODate(“2020-07-13T22:18:12.215Z”), “index_analyzer” : “standard”, “index_template_name” : “gl-events-template”, “index_template_type” : “events”, “index_optimization_max_num_segments” : 1, “index_optimization_disabled” : false, “field_type_refresh_interval” : NumberLong(60000), “writable” : true }
{ “_id” : ObjectId(“5f0cdda419bb63000c19263c”), “title” : “Graylog System Events”, “description” : “Stores Graylog system events.”, “index_prefix” : “gl-system-events”, “shards” : 4, “replicas” : 0, “rotation_strategy_class” : “org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy”, “rotation_strategy” : { “type” : “org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig”, “rotation_period” : “P1M” }, “retention_strategy_class” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy”, “retention_strategy” : { “type” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig”, “max_number_of_indices” : 12 }, “creation_date” : ISODate(“2020-07-13T22:18:12.330Z”), “index_analyzer” : “standard”, “index_template_name” : “gl-system-events-template”, “index_template_type” : “events”, “index_optimization_max_num_segments” : 1, “index_optimization_disabled” : false, “field_type_refresh_interval” : NumberLong(60000), “writable” : true }
Streams
rs0:PRIMARY> db.streams.find( {} )
{ “_id” : ObjectId(“000000000000000000000002”), “creator_user_id” : “admin”, “is_default_stream” : false, “index_set_id” : “5f0cdda419bb63000c192639”, “matching_type” : “AND”, “remove_matches_from_default_stream” : true, “description” : “Stream containing all events created by Graylog”, “created_at” : ISODate(“2020-06-18T09:53:51.749Z”), “disabled” : false, “title” : “All events” }
{ “_id” : ObjectId(“000000000000000000000001”), “creator_user_id” : “local:admin”, “is_default_stream” : true, “index_set_id” : “5eeb39a35e9f31000cb2f4b1”, “matching_type” : “AND”, “remove_matches_from_default_stream” : false, “description” : “Stream containing all messages”, “created_at” : ISODate(“2020-06-18T09:53:39.443Z”), “disabled” : false, “title” : “All messages” }
{ “_id” : ObjectId(“000000000000000000000003”), “creator_user_id” : “admin”, “is_default_stream” : false, “index_set_id” : “5f0cdda419bb63000c192639”, “matching_type” : “AND”, “remove_matches_from_default_stream” : true, “description” : “Stream containing all system events created by Graylog”, “created_at” : ISODate(“2020-06-18T09:53:51.845Z”), “disabled” : false, “title” : “All system events” }
{ “_id” : ObjectId(“5f06e08c3f1b79000dc1d44a”), “creator_user_id” : “admin”, “index_set_id” : “5f0cdd9719bb63000c1925a4”, “matching_type” : “AND”, “remove_matches_from_default_stream” : false, “description” : “All things ETL related”, “created_at” : ISODate(“2020-07-09T09:17:00.697Z”), “disabled” : false, “title” : "ETL ", “content_pack” : null }
administrator@Ranchercluster01:~$ rs0:PRIMARY> db.index_sets.find( {} )
{ “_id” : ObjectId(“5f0cdda419bb63000c19263c”), “title” : “Graylog System Events”, “description” : “Stores Graylog system events.”, “index_prefix” : “gl-system-events”, “shards” : 4, “replicas” : 0, “rotation_strategy_class” : “org.graylog2.indexer.rotat-bash: syntax error near unexpected token `(’
ion.strategies.TimeBasedRotationStrategy”, “rotation_strategy” : { “type” : “org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig”, “rotation_period” : “P1M” }, “retention_strategy_class” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy”, “retention_strategy” : { “type” : “org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig”, “max_number_of_indices” : 12 }, “creation_date” : ISODate(“2020-07-13T22:18:12.330Z”), “index_analyzer” : “standard”, “index_template_name” : “gl-system-events-template”, “index_template_type” : “events”, “index_optimization_max_num_segments” : 1, “index_optimization_disabled” : false, “field_type_refresh_interval” : NumberLong(60000), “writable” : true }
thanks
Peter