Question about configuration of OpenSearch cluster

[peaile (CéPL)]

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
I currently want to deploy a cluster with OpenSearch like this :

• 1 master node (“master-1”, 192.168.40.136)
• 2 data nodes (“data-1”, 192.168.40.137 / “data-2”, 192.168.40.139)
  I don’t understand how to fill in the various parameters in the config file especially what to put in discovery.seed_hosts (should it be all the adress of nodes, including this on which I am configuring OpenSearch) and cluster.initial_cluster_manager_nodes. I find that the official documentation don’t explain this really well and all the tutorial are on Elasticsearch, which will be deprecated.

When I want to try to interact with OpenSearch, after the configuration, on one of my nodes like this : curl 192.168.40.136:9200
I got the following output :
curl: (52) Empty reply from server

However, while checking the status with systemctl status opensearch, all the service are active (but only the master is enabled)

2. Describe your environment:

• OS Information:
  Debian 12

• Package Version:
  2.15.0

• Service logs, configurations, and environment variables:
  Here is the config files of my 3 nodes (expect experimental features which are unchanged).
  master-1 :

cluster.name: myCluster
node.name: master-1
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 192.168.40.136
http.port: 9200
discovery.seed_hosts: ["192.168.40.136", "192.168.40.137", "192.168.40.139"]
cluster.initial_cluster_manager_nodes: ["master-1", "data-1", "data-2"]

data-1 :

cluster.name: myCluster
node.name: data-1
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 192.168.40.137
http.port: 9200
discovery.seed_hosts: ["192.168.40.136", "192.168.40.137", "192.168.40.139"]
cluster.initial_cluster_manager_nodes: ["master-1", "data-1", "data-2"]

data-2 :

cluster.name: myCluster
node.name: data-2
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 192.168.40.139
http.port: 9200
discovery.seed_hosts: ["192.168.40.136", "192.168.40.137", "192.168.40.139"]
cluster.initial_cluster_manager_nodes: ["master-1", "data-1", "data-2"]

3. What steps have you already taken to try and solve the problem?
Basically all the configurations that I can find on internet but it don’t work so I guess i’ve made an error in my implementation

4. How can the community help?
I guess it’s a really little error but if someone can light me on the origin of the error
Feel free to ask me for more details if you need it

[Joel_Duffield]

First question, why do you want a dedicated master, that normally isnt required until much larger clusters.

[Sinan]

I’m working on a similar setup. in my case, the issue is that I get “connection refused” error from master node in regards to the 9200 port of data node. Same issue when I use curl. @peaile I would recommend two things:

1. grep ‘ERROR’ in log files
2. indicate port number of mongodb in config file:
   mongodb_uri = mongodb://<ip.of.master.node.here>:27017/graylog
   I believe mongodb_uri needs to be indicated in both master config and datanode config.
   update. I realized that indicating the default port number is not necessary. The only requirement is to make sure bindIp: 0.0.0.0 in /etc/mongod.conf

[Sinan]

in my case, graylog docs recommend 2x datanodes according to the amount of logs processed per day (25GB/day).

Planning Your Deployment

[Joel_Duffield]

Ah, yes so two is fine, but they can be shared masters, so all nodes run all the roles, so just two nodes total, and yes generally for most of those settings like seed hosts i will just list all the opensearch nodes, because in this case they are all the same.

You only need dedicated masters when you get into the range of like 5+ nodes.

[Sinan]

Thanks for the quick reply. I am kinda inclined to having graylog + mongo vs datanode on separate machines. That’s also what’s advised in installation instructions:

Hint: For production environments, we do not recommend installing and running Data Node and Graylog server on the same machine.

Would appreciate any hints for this setup. In docs for graylog multinode setup, there’s no mention of using graylog datanode, hence there’s a bit of ambiguity I think.

[Joel_Duffield]

Datanode and opensearch are interchangable in any of these architecture discussions. So yes best to put graylog and mongo on the same box, and then datanode/opensearch on another.

[Sinan]

Now, my setup is working thanks to a redit comment. Agree with @peaile that documentation is confusing. In particular, in ‘Get Started with Graylog’ → "initial config’ section, user’s are told to have ‘elasticsearch_hosts’ list in the config. Because of which I wasted two days. As soon as, I commented it out, graylog was able to find the datanode and issue the cert etc. At first glance, my setup seems to be working.

Initial Configuration Settings

[peaile (CéPL)]

Hello Joel thanks for your reply

I’m currently planning to deploy a cluster because i want to centralize the logs from my Stormshield firewalls and from Active Directory, which are numerous. As I begin, I was trying to form a very small cluster just to start manipulating.

However i find the issue.
By analyzing the logs i discovered that the problem was my SSL/TLS certificate, which was not configured.
I added this line to avoid this verification :

plugins.security.disabled: true

For the moment I only connect myself with HTTP but it’s not annoying.

Also my configuration was correct and I think it will be great to let this topic for someone like me who is starting and find that the doc of OpenSearch is unclear.

Have a nice day !