I’m testing my ability to run Graylog on Docker Compose using named volumes and then migrate to another Docker Compose host. The compose file spins up the following services and volumes:
Graylog, with volumes “graylog-data” and “graylog-journal”
Graylog Datanode, with volume “graylog-datanode”
MongoDB, with volume “mongodb-data”
My order of operations is to export the volumes to .tar from the old host, spin up the application on the new host, extract the .tar files into the volumes on the new host, and restart all services. And although I can see the files in the volumes on the new host, absolutely nothing is picked up by the application—no logs, no certificates, no configuration data.
Variables that I have tested inconclusively include lock files in MongoDB and the data node, the order-of-operations for deploying certificates, and the node IDs. Any one of these—or none of them—may yet be the culprit. Does anyone have any insight or guidance?
Both hosts running in the test environment are identical: Ubuntu 20.04, Docker 20.10.24, Graylog 5.2, and MongoDB 5.0.
Can you clarify a but more what this means? What did you move and what are you not seeing? Did you move ALL of the data volumes you listed above?
Also can you confirm you moved the data in a support manner e.g. Volumes | Docker Docs ?
My recommendation is to move/test each container individually.
For example, starting with mongo, moving the data and started the new container, is the data available when querying via mongo? (you may need to use a tool such as Studio 3T.
I applogize that i can’t help with datanode as its not been released as GA (generally availalbe and is still technically experimental)
I appreciate the response. Yes, I migrated all the volumes, and what I mean by “not picked up” is that the application behaves as if it’s been spun up afresh by Docker Compose. None of the configuration nor accumulated logs, etc. from the first host are evident.
I’m pretty fluent with Docker volume migration practices, and I did try migrating just the MondoDB volume with the same result. I will try querying Mongo directly as you suggested and see what that tells me.
I sorted out the issue. My fault—something I should have seen. When restoring the volumes, I was inconsistent with the tar --strip parameter and the Mongo volume restored to the wrong directory. For example, this:
docker run --rm --volumes-from 557b6b76b532 -v $(pwd):/backup busybox sh -c "cd /data/db && tar xvf /backup/mongodb-data.tar --strip 1"
…is incorrect because it will extract to /data/db/db/ in the container. Since the volume is mounted two directories deep, I needed to revise to --strip 2.
And of course the other volume container paths should be double-checked accordingly when restoring to the new host:
docker run --rm --volumes-from de0f9b910c7f -v $(pwd):/backup busybox sh -c "cd /usr/share/graylog/data/data && tar xvf /backup/graylog-data.tar --strip 5"
docker run --rm --volumes-from de0f9b910c7f -v $(pwd):/backup busybox sh -c "cd /usr/share/graylog/data/journal && tar xvf /backup/graylog-journal.tar --strip 5"
docker run --rm --volumes-from 811ff5580d0f -v $(pwd):/backup busybox sh -c "cd /var/lib/graylog-datanode && tar xvf /backup/graylog-datanode.tar --strip 3"
