@gsmith Thanks for the info ! I think we were going to stick to the Elasticsearch 6.8. Is there a need to switch to Opensearch vs Elsaticsearch?
@ihe Unfortunately because the saved searches didn’t migrate from 3.1 to 3.2, we don’t have a way to import them using the content pack because the saved searches is not available as a content pack sources in 3.1.
While doing some more poking around on graylog MongoDB, we found an entry in the cluster_config collection that had the saved_searches migration listed as complete. The migration for the dashboard failed before, so we rolled back to 3.1, which we have been using for over 2 years now.
We restored selective collections from the MongoDB dump and excluded the cluster_config collection. Here’s the list of the collections we included in the mongorestore.
inputs
roles
users
ldap_settings
dashboards
saved_searches
searches
event_processor_state
alerts
streamrules
index_sets
notifications
aggregate_rules
alarmcallbackconfigurations
aggregate_report_schedules
streams
index_field_types
access_tokens
event_definitions
event_notifications
sessions
Here are the mongorestore commands:
mongorestore --host $MONGODB_HOST --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD --nsInclude=graylog.dashboards --nsInclude=graylog.saved_searches --nsInclude=graylog.searches --nsInclude=graylog.event_processor_state graylog-mongo-backup/
mongorestore --host $MONGODB_HOST --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD --nsInclude=graylog.alerts --nsInclude=graylog.streamrules --nsInclude=graylog.index_sets --nsInclude=graylog.notifications graylog-mongo-backup/
mongorestore --host $MONGODB_HOST --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD --nsInclude=graylog.aggregate_rules --nsInclude=graylog.alarmcallbackconfigurations --nsInclude=graylog.aggregate_report_schedules --nsInclude=graylog.streams graylog-mongo-backup/
mongorestore --host $MONGODB_HOST --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD --nsInclude=graylog.index_field_types --nsInclude=graylog.access_tokens --nsInclude=graylog.event_definitions --nsInclude=graylog.event_notifications graylog-mongo-backup/
After performing the upgrade from 3.1 to 3.2, the migration was successful and everything showed up as expected.
It appears our issues was related to cluster_config collection having incorrect data which didn’t trigger another migration, even though we rolled back our deployment from the failed MongoDB migration.
I think we are all good here and added details from what we experienced so others can quickly move forward if they encounter a similar issue.