Hello everyone,
I have cluster with 3 graylog servers (elasticsearch and mongodb are on each server). I configured 3 shards and 1 replica, this is minimal setup for now. In the Graylog Index view, I see al shards are green and the idea is 1 shard per node.
When 1 node goes down, the replica shard of the offline server automatically becomes the primary shard which is good. But when the server returns, the replica shard stays as original, while the previous primary shard is replica now. In that case, I have 2 primary shards from node1 for example, and 1 from node2, while both shards of node3 are set as replicas. Is this normal or shouldn’t be like that?
To ilustrate:
BEFORE NODE3 WENT DOWN:
Node1 - S0 (primary - node1), S0 (replica - node2)
Node2 - S0 (primary - node2), S0 (replica - node3)
Node3 - S0 (primary - node3), S0 (replica - node1)
AFTER IT GOES UP:
Node1 - S0 (primary - node1), S0 (replica - node2)
Node2 - S0 (primary - node2), S0 (replica - node3)
Node3 - S0 (primary - node1), S0 (replica - node3)
By this logic, Node3 will never get any data.
Thanks in advance