Shrink 2-node graylog cluster to 1 (graylog@6.1)

Graylog Central (peer support)
1

**1. Describe your incident:**

Trying to go from 2 datanodes to 1. Let’s label datanodes as dn1 & dn2. all shards are located to dn2 via an API call. When I click “Stop” for dn1 on web UI, graylog UI seems to work ok but throws this error: cluster_manager_not_discovered_exception . current manager is dn1, it seems whatever I do, dn2 isn’t able to assume the cluster manager role.

2. Describe your environment:

  • OS Information: Linux

  • Package Version: 6.1

  • Service logs, configurations, and environment variables:
    configured: shards 1, replicas:0.

**3. What steps have you already taken to try and solve the problem?**

  • tried to stopping datanode service on the master dn1, but dn2 couldn’t acquire manager role.
  • I can’t remove dn1, before making sure graylog in good state.

**4. How can the community help?**

appreciate any help on this. If and how can I make dn2 manager? Would it help to add a new but minimal datanode to help with cluster quorum?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

Hey @Sinan,

You try removing voting rights from the node you want to remove before removing it.

POST /_cluster/voting_config_exclusions?node_names=node-2

In theory once the nodes voting rights have been removed, if it is leader that should be transfered to the new node.