Thanks both,
Yea I have checked the ES logs. I only have INFO logs for a second node not connecting (currently setting up a second node )
[2019-03-27T09:02:06,057][INFO ][o.e.d.z.ZenDiscovery ] [grayone] failed to send join request to master [{crow}{KQtVL-x1TbWLGcvDiGyXLA}{4_Ae58sDRty8Dwa6i-Zouw}{172.24.228.226}{172.24.228.226:9300}{ml.machine_memory=67529670656, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[crow][172.24.228.226:9300][internal:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[grayone][172.24.228.161:9300][internal:discovery/zen/join/validate]]; nested: IllegalArgumentException[Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]; ]
But this wouldn’t effect the main node?
Yea checked the disk space and it has plenty there and I have set cluster.routing.allocation.disk.threshold_enabled: false in the ES conf
Filesystem Size Used Avail Use% Mounted on
udev 32G 0 32G 0% /dev
tmpfs 6.3G 1.4M 6.3G 1% /run
/dev/sda2 246G 94G 140G 41% /
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 32G 0 32G 0% /sys/fs/cgroup
tmpfs 32G 0 32G 0% /run/shm
/dev/loop0 91M 91M 0 100% /snap/core/6405
/dev/loop1 87M 87M 0 100% /snap/core/4486
/dev/loop2 12M 12M 0 100% /snap/lnav/116
/dev/loop3 92M 92M 0 100% /snap/core/6531
tmpfs 6.3G 0 6.3G 0% /run/user/1000
Checking ES health it returns
{
“error” : {
“root_cause” : [
{
“type” : “master_not_discovered_exception”,
“reason” : null
}
],
“type” : “master_not_discovered_exception”,
“reason” : null
},
“status” : 503
}
Which is odd because
is_master = true
is set in the graylog.conf and just a basic conf for ES (below)
cluster.name: graylog
action.auto_create_index: false
cluster.routing.allocation.disk.threshold_enabled: false
network.host: 172.24.228.161
node.name: grayone
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
discovery.zen.ping.unicast.hosts: [“172.24.228.161”,“172.24.228.226”]
I have RTFM, but its not specific and a bit vague with the ES set up. Not sure where to go from here?