I installed graylog more a year ago from OVA image. In the meantime I did several updates successfully. I have now Graylog 2.5.1+34194da on graylog (Oracle Corporation 1.8.0_191 on Linux 4.4.0-97-generic)
with DB : /opt/graylog/elasticsearch/bin/elasticsearch --version
Version: 5.6.13, Build: 4d5320b/2018-10-30T19:05:08.237Z, JVM: 1.8.0_191
2.x It is not possible to upgrade previous OVAs to Graylog 3.0.0.
( even I tried, but of course I failed )
Actually a clear statement. But is this a general statement or is simple an automated upgrade not possible ?
If manually possible, is there a description anywhere available ? This doc http://docs.graylog.org/en/3.0/pages/upgrade/graylog-3.0.html#
describes partly the changes made in the new version and new features it but didn’t give a step-by-step how to do.
Of course I could setup V 3 from OVA again but my “old” graylog got a lot of changes over time. And there are still some questions:
is there a way to take over all configurations, like all inputs, stream and alerts ?
what about to take over elasticsearch DB with all the logs from last half year ?
Actually a clear statement. But is this a general statement or is simple an automated upgrade not possible ?
that is a general statement - hacker solutions are possible. Like exporting MongoDB and Elasticsearch off the OVA, take the new OVA and import the data.
But not supported and much knowledge about the different products are needed.
Recently I ended succesfull migration from Graylog 2.5 (virtual appliance) to 3.0 (installed from packages). I had about 6 months of logs in total about only 30 GB. It took a lot of head scratching and googling. My servers are virtual, so a lot of snapshots before every step. It helped a lot, needed some rollbacks.
Installing and initial configuring was almost easy, documentation is good in this part.
The hardest part was Elasticsearch data migration. I had no experience in Elasticsearch, so learned a lot. There are some tools on Internet for ES data migration, none worked for me, so I ended with snapshotting and restoring from snapshot. This could help: Make Snapsots of Elasticsearch Data and Restore It, and perhaps this too: Talking to Elasticsearch.
Elasticsearch v.6 command ‘curl -XPUT’ with JSON content should contain ‘content-type: application/JSON’ in header, else command will end with error. Example: curl -H "content-type: application/JSON" -XPUT 'http://localhost:9200/_snapshot/graylog_backup' -d ' { "type": "fs", "settings": { "compress" : true, "location": "/backups/elasticsearch-backup" } }
There were problem with Active Directory authentication after migration. Login with AD user not worked, settings were not shown in Web interface, but all users are there. Tried to input AD authentication settings, they were not saved. From server log found, there is one configuration already and multiple LDAP backends are not supported. Cleared previous settings in mongodb database, now it was possible to save new configuration. Solution found here: LDAP Settings - Not Saving
Many thanks for your reply letting us know there is a way to migrate. It’s the question how much effort and time you investigated ? Of course you learned a lot for ES and MongoDB and this is a win-situation for you.
I started to install a new server with version 3.0 from OVA image in the meantime. Installation for this new VM is done very fast. After about 5 hours of configuration the new one is at least collecting information from all the clients as the old one has done it. But I would guess I need at least 20 additional hours until the “new” one has the same config as the “old” one. For example I still miss the predefined search strings.
Finally many thanks to the graylog team for providing us an OVA image which is really easy to setup.
Migration process took about 8 hours, plus time to copy data from old server to new. As I wrote, it includes thinking, googling, coffee drinking and shouting bad words
You’re right, it’s not very hard. However, when you start taking things really seriously and you start building clusters, then it can get a bit complicated. I’ve written a runbook for my colleagues that goes through the process of building a full Graylog stack cluster, with all the clustering, certificates, security and configuration, which spans 40 pages. Took me about a week’s work to go from zero to finished on that.
Good day.
Someone help me please.
I’m trying to migrate from 2.5.1 to 3.0.0, VA.
Stucked at elasticsearch snapshot restoring on machine with graylog 3:
{“error”:{“root_cause”:[{“type”:“repository_exception”,“reason”:"[backup] could not read repository data from index blob"}],“type”:“repository_exception”,“reason”:"[backup] could not read repository data from index blob",“caused_by”:{“type”:“access_denied_exception”,“reason”:"/var/backups/elasticsearch/pending-incompatible-snapshots-b-_QN6l5SaG8Ed9h2AjgzA"}},“status”:500}
Snapshot files just copy-pasted to /var/backups/elasticsearch.
Snapshot was done by the following command:
I also thought about permissions but was embarrassed by the phrase: “reason”:"/var/backups/elasticsearch/pending-incompatible-snapshots-b-_QN6l5SaG8Ed9h2AjgzA"
I’ll try, thx
