I am on the latest versions of Graylog Open and Graylog Datanode in a 3 node setup. I am attempting to do an inplace upgrade from Opensearch 2.15 to Datanode. The migration process keeps failing at various steps.
The first place the migration will fail is on step one. After a few attempts I seem to be able to get past that step by stopping graylog server and datanode, restarting opensearch and then starting graylog again on each node. I used the Cluster Global API to trouble shoot this from this [link] (Be a part of Graylog Development - Call for testers! - #7 by matthias_gl). Once that came back ok I was able to click “Run directory compatilbilty check”.
It then failed at step 3, provision data node certificate. I tried manually updating the certificate using the Datanodes > Actions > More > Renew certificate, which did renew the certificate but I still get stuck on step 3. I tried messing with opensearch permissions as detailed here but then opensearch would not start so I reverted that permission.
I am now stuck back at step 2. I have tried adding graylog and graylog-datanode to the opensearch group but that doesn’t seem to have any affect.
Anyone have any ideas? I have a feeling I am running in to permission issues with Graylog and Opensearch but that is just a theory at this point.