We have Graylog and Elasticserach running in what I think is a pretty simple setup. I was making some adjustments to the iptables NAT engine and rebooted the server. Graylog will now not start. The log found at:
/var/log/graylog-server/server.log
…is saying…
################################################################################
ERROR: Unable to probe any host for Elasticsearch version!
Please see the following link(s) to help you with this error:
*
Need further help?
This server has been running for a few months with almost no problems. I don’t know what has changed to create this new behavior.
Description of steps you’ve taken to attempt to solve the issue
I have restarted Graylog…
sudo systemctl restart graylog-server
And I have restarted MongoDb…
sudo service mongod restart
And I have restarted the entire server.
Restarting Elasticsearch produces the following error:
$ sudo systemctl restart elasticsearch.service
Job for elasticsearch.service failed because the control process exited with error code.
See “systemctl status elasticsearch.service” and “journalctl -xe” for details.
the issue is that Graylog is probing for the ES version used on startup. It goes through the list of configured nodes and gives up if none of them are up. Can you make sure that ES is available before GL is starting up.
If you need help with elasticsearch please post your full ES YAML file and ES log file.
Also, if you can post the following command that would be great.
Hey there, you should be able to specify the elasticsearch version using elasticsearch_version = <YOURVERSIONNUMBER> in your server.conf file and restart graylog.
Thank you for your help. It was ES that was the problem. Because of recent changes by me, it did not have access to /var/lib/elasticsearch. I think I did those changes back on Wednesday, so I think it didn’t affect ES on Wednesday because it already had the files open, so the change didn’t kick it out immediatly. When I rebooted the server on Friday it tripped it up.
Fixed it with this… chown elasticsearch:elasticsearch /var/lib/elasticsearch