Graylog 7.0.11 with self-manged Opensearch 2.19.3 cluster, looking to migrate to Graylog Data Node

Graylog Central (peer support)
1

1. Describe your incident:

Hi guys!

I’m in the process of planning to migrate our self-managed OpenSearch backend to Graylog Data Nodes. We currently have 3 OpenSearch master nodes (VM’s) and 4 Data Nodes (physical servers). TLS is enabled via security plugin. After having read the migration documentation I’ve got two questions that I can’t find the answer for.

2. Describe your environment:

  • OS Information: Deb 12, upgrade to Deb 13 is planned for the near future

  • Package Version: graylog-server 7.0.11-1

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

4. How can the community help?

  1. The migration documentation doesn’t say anything about Master nodes, only the following:
  • Your Data Node installation will be on the same hosts as existing OpenSearch nodes.

Since Master nodes do not contain any data, and management of nodes is built into Graylog Data Node, I’m assuming the Master nodes will be just made redundant during the migration? At the OpenSearch service shutdown step, I’m just gonna shut down the Master nodes and leave them be, only install Graylog Data Node on the 4 OpenSearch Data nodes?

  1. In addition, the migration documentation under conditions states, that
  • You have deployed Graylog directly on a physical host, not within containers or on virtual hosts, using the same parameters outlined in the installation documentation.

Is this a hard requirement? Or just the recommended/supported migration path? The installation documentation that is linked in that bulletpoint only says the following:

There are two primary deployment methods for installing Graylog: directly on a Linux-based system or in a containerized environment with Docker Compose

Wouldn’t VM’s fall under the first category? Currently we are running a Graylog cluster with 3 nodes, all on Proxmox VM’s, without any problems. So I’m assuming that the in-place TLS secured upgrade can be performed without problems as well. Or should I test it out beforehand?

Thank you!

2

Hi @ovreba,

Good questions!

  1. you can replicate the same topology with datanodes. You’ll still need master/manager nodes with datanode. Datanode is mainly a process management wrapper around the opensearch itself, it doesn’t replace its clustering functionalities. I’d recommend installing datanode on your 4 master nodes and in the datanode.conf set `node_roles=cluster_manager`. This will replicate your current topology.

  2. datanodes will run fine in virtualized environments, this is not hard requirement and the quote serves just for documentation purposes - to clarify that described commands and configuration are valid only for usual setups. You’ve been running Opensearch in your virtual machines, so you know how to setup, configure, access data directories, manage processes. Feel free to do the same with datanodes.

Let me know if you have more questions or hit any problems.

Best regards,
Tomas