Opensearch Cluster

Graylog Central (peer support)
1

Hi everyone !

I’m planning on creating a new Graylog production architecture with OpenSearch on a Debian based OS. But I’m no OpenSearch expert. From my understanding, one should use OpenSearch the same way ElasticSearch. Meaning you declare them this way in the server.conf

elasticsearch_hosts = http://es-node-1.example.org:9200/foo,https://someuser:somepassword@es-node-2.example.org:19200

My questions are :

  • how to manage role assignment in the cluster ?
  • Should I use the kind of setup shown in the Creating a cluster documentation of Elasticsearch ?
  • Should I declare coordinating et manager nodes in the previous list ?
  • Should I even have a manager and a manager node ?
  • Should all nodes be master eligible ?
  • Can I use a three nodes cluster, like I did with Elasticsearch ?

Thanks everyone ! :cowboy_hat_face:

2

Greetings! All very good questions. The short answer is OpenSearch behaves nearly identical to Elasticsearch, because it is a fork.

Regarding cluster configuration and formation info, OpenSearch have a document covering this via: Creating a cluster - OpenSearch documentation

This explains node/role types and how to configure them. My understanding of the recommendation for OpenSearch nodes is that unless you are sending a lot of log data (>550GB/day) and you have several opensearch nodes, you won’t need dedicated masters, and shouldn’t need to explicitly define server roles.

To more clearly answer all your questions:

  • how to manage role assignment in the cluster ?
  • Should I use the kind of setup shown in the Creating a cluster documentation of Elasticsearch ?
    • see above
  • Should I declare coordinating et manager nodes in the previous list ?
    • Generally no
  • Should I even have a manager and a manager node ?
    • Generally no
  • Should all nodes be master eligible ?
    • Generally yes
  • Can I use a three nodes cluster, like I did with Elasticsearch ?
    • Yes.

Hope that helps. Let us know if you still have any questions or need any clarifications.

3

Allright, thanks for the answer !

So, to use this kind of production setup for less than 550GB/day.

  • Create three Opensearch nodes
  • Make each nodes master eligible
  • Declare them in server.conf the same way as you would declare elasticsearch_hosts.
  • Enjoy ?

What confused me was that in Creating a cluster - OpenSearch documentation, they showcase a 4 nodes architectures, with a cluster manager node, a coordinating node and data ingesting nodes. I was not sure how to make this fit with a Graylog production setup.