"no leader Graylog server node" alert toggles on/off every nth second

Check the publish_uri in server.conf of every graylog node, this needs to be the address, either ip or fqdn of that machine that the other machines can use to reach it.