Restore MongoDump in Graylog

Graylog Central (peer support)
1

1. Describe your incident:
I currently have the problem, that the hypervisor of my current Graylog VM crashed, so I had to setup a new VM. Luckily I had MongoDB dumps, so I wanted to restore all the data. On the new VM I therefore dropped the graylog table and ran mongorestore to restore my old data. The only problem is, that in Graylog I then get an Elasticsearch error

Elasticsearch exception [type=index_not_found_exception, reason=no such index []].

To be able to display new logs, I then deleted all indices, which worked in the sense, that new logs are now displayed again

curl -X DELETE 'http://localhost:9200/_all'

However, long-term, I would like to be able, to have a visual representation of the old logs (from the dump) aswell. I’d much appreriate any help.

2. Describe your environment:

  • OS Information: Debian 11.5, Kernel 5.10.0-9-amd64

  • Package Version: 4.3.8+8c4705e

  • Java-Version:
    openjdk 11.0.16 2022-07-19
    OpenJDK Runtime Environment (build 11.0.16+8-post-Debian-1deb11u1)
    OpenJDK 64-Bit Server VM (build 11.0.16+8-post-Debian-1deb11u1, mixed mode, sharing)

  • MongoDB Version:
    MongoDB shell version v4.2.23
    git version: f4e6602d3a4c5b22e9d8bcf0722d0afd0ec01ea2

  • Elasticsearch Info:

{

  "name" : "hostname",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "****",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "****",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

  "cluster_name" : "elasticsearch",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 16,
  "active_shards" : 16,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 6,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 72.72727272727273
}```
2

Hello && welcome @mlhb
I see you were unable to save your old logs with restoring MongoDb. I have done this a couple times and received the same/similar error. It has been a a few years ago but my personal documents had a fix, this may or may not help but ill share it with you.

The basic steps I used I shorten it a bit for a general description.

sudo mongodump --db graylog --out /home/mongobackup/`date +"%m-%d-%y"` -u mongo_admin

Copy all the configuration files, certificates, etc… i.e., graylog, Elasticsearch & MongoDb).

curl -X POST "127.0.0.1:9200/_snapshot/my_repo/snapshot_1/_restore?pretty

Execute mongorestore dump/ and service is working.
Once ES repo is migrated ensure Elasticsearch status is in “Green” .

If an errors do occur check indices.

ERROR [IndexRotationThread] Couldn’t point deflector to a new index
java.lang.IllegalArgumentException: [alias] is unsupported for [REMOVE_INDEX]

Elasticsearch exception [type=index_not_found_exception, reason=no such index ].

curl -XGET 'http://127.0.0.1:9200/_cat/indices?pretty=true'

If there are old indices make sure to remove them.
Example if I restored index graylog_1112 and if is index name graylog_0 or graylog_1 you need to remove them like this.

curl -XDELETE 127.0.0.1:9200/graylog_0

Note:

Rotating the indices manually on each index set.

When Migrating or just installing Graylog. Elasticsearch/OpenSearch is my first priority for starting services, then MongoDb and last start graylog service. When each service is started I also check status and logs files before continuing.

Here is another example of this that may help.

MongoDb does not hold Logs it retains metadata. Basically any configuration that were made. Elasticsearch is where you old logs are stored so if you do get an error like " no index found" you may have to make that index through Graylog GUI and/or MongoDb.

hop that helps

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.