# Down cluster with 3 node **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [September 29, 2021, 12:26pm UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325 "2021-09-29T12:26:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Uporaba](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/uporaba/32/11232_2.png) [@Uporaba](https://community.graylog.org/u/Uporaba)\ **Post date:** [September 29, 2021, 12:26pm UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/1 "2021-09-29T12:26:27Z")
Hello. I have a cluster of 3 nodes with elastic (2 node with graylog, and 1 single node with elastic), when one falls, the entire cluster fails (and becomes red). How can I fix this so that when one node fails, the rest work? My elastic settings: ```auto cluster.name: graylog node.name: es-node-01 #node.attr.rack: r1 path.data: - /var/lib/elasticsearch/data1 - /var/lib/elasticsearch/data2 path.logs: /var/log/elasticsearch #bootstrap.memory_lock: true network.host: 10.101.15.231 http.port: 9200 discovery.seed_hosts: ["10.101.15.231", "10.101.15.232", "10.101.15.233"] discovery.zen.fd.ping_timeout: 10s discovery.zen.minimum_master_nodes: 2 cluster.initial_master_nodes: - es-node-01 - es-node-02 - es-node-03 #gateway.recover_after_nodes: 3 #action.destructive_requires_name: true action.auto_create_index: false node.roles: [data, master] cluster.routing.allocation.disk.watermark.low: "20G" cluster.routing.allocation.disk.watermark.high: "10G" cluster.routing.allocation.disk.watermark.flood_stage: "5G" path.repo: - /var/lib/elasticsearch/data1/backup ``` ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [September 29, 2021, 11:00pm UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/2 "2021-09-29T23:00:21Z")
Hello, By chance is this related to this post. > [@Elasticsearch exception reason=all shards failed](https://community.graylog.org/t/elasticsearch-exception-reason-all-shards-failed/21315): > > hi, in the new version 4.1.3, this error appears. How can I understand what this error is and remove it? I don’t see any errors in the logs. but in graylog version 3.2.4 everything is searched and there are no errors: UPD: In log i saw: 2021-09-28T07:50:04.217Z ERROR [MessagesAdapterES7] Failed to index [3] messages. Please check the index error log in your web interface for the reason. Error: failure in bulk execution: [150]: index [graylog\_57], type [\_doc], id [b0d… As for you question. > [@Uporaba](#): > > How can I fix this so that when one node fails, the rest work? My elastic settings: You might want to read this documentation on Multi-node Setup. [https://docs.graylog.org/en/4.1/pages/configuration/multinode\_setup.html#configure-multinode](https://docs.graylog.org/en/4.1/pages/configuration/multinode_setup.html#configure-multinode) Also this documentation is very helpful. > **[Designing for resilience | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-design.html)** Hope that helps ---
**Author:** ![Uporaba](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/uporaba/32/11232_2.png) [@Uporaba](https://community.graylog.org/u/Uporaba)\ **Post date:** [September 30, 2021, 6:08am UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/3 "2021-09-30T06:08:44Z")
> [@gsmith](#): > > By chance is this related to this post. This is my post, but they are not related in any way, these are two different tasks > [@gsmith](#): > > You might want to read this documentation on Multi-node Setup. I read the documentation, but for some reason nothing works. I made the elastic cluster setup as shown in the [official documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-small-clusters.html#high-availability-cluster-design-three-nodes) (if I understood it correctly): ```auto cluster.name: graylog node.name: es-node-01 #node.attr.rack: r1 path.data: - /var/lib/elasticsearch/data1 - /var/lib/elasticsearch/data2 path.logs: /var/log/elasticsearch #bootstrap.memory_lock: true network.host: 10.101.15.231 http.port: 9200 discovery.seed_hosts: ["10.101.15.231", "10.101.15.232", "10.101.15.233"] discovery.zen.fd.ping_timeout: 10s discovery.zen.minimum_master_nodes: 2 cluster.initial_master_nodes: - es-node-01 - es-node-02 - es-node-03 #gateway.recover_after_nodes: 3 #action.destructive_requires_name: true action.auto_create_index: false node.roles: [data, master] cluster.routing.allocation.disk.watermark.low: "20G" cluster.routing.allocation.disk.watermark.high: "10G" cluster.routing.allocation.disk.watermark.flood_stage: "5G" path.repo: - /var/lib/elasticsearch/data1/backup ``` the configuration is the same on all nodes ---
**Author:** ![Uporaba](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/uporaba/32/11232_2.png) [@Uporaba](https://community.graylog.org/u/Uporaba)\ **Post date:** [September 30, 2021, 6:40am UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/4 "2021-09-30T06:40:30Z")
Is solved. I just needed add replics in index. ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [October 1, 2021, 2:57am UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/5 "2021-10-01T02:57:24Z")
Nice, and thanks for posting how you fixed it. This will help other down the road that may have the same problem/s. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [October 15, 2021, 2:57am UTC](https://community.graylog.org/t/down-cluster-with-3-node/21325/6 "2021-10-15T02:57:36Z")
Closed on October 15, 2021, 2:57am UTC This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.