Hi All
I feel silly asking this question but I have this this situation:
I am using an external elastic search server (actually a hosted AWS Elastic search)
This then overwrites the elasticsearch_hosts setting in the server.conf to point to the local ip address.
Is there a way to set the elasticsearch_hosts so that graylog-ctl reconfigure does not overwrite it?
hej @michaelwiles
the AMI Image is not build to run with AWS Elasticsearch. Please refer to: http://docs.graylog.org/en/2.3/pages/configuration/graylog_ctl.html
You would need to build your own setup to support AWS Elasticsearch.
I have most certainly done that. I have trawled those logs for the directive and I cannot find any.
I have run sudo graylog-ctl reconfigure-as-server to reconfigure my graylog to not run elasticsearch.
So I have been able to get it working by manually setting the graylog.conf file to point to my external elasticsearch.
What I’m after is if it’s possible somehow to set the elasticsearch_hosts so that graylog-ctl reconfigure does not overwrite it. Say by editing graylog-settings.json to include the elasticsearch_hosts property. All the docs say is set the elasticsearch_hosts property. They don’t tell you where to set it.
please refer to the advanced settings: http://docs.graylog.org/en/2.3/pages/configuration/graylog_ctl.html#advanced-settings
And all possible settings referenced here: https://github.com/Graylog2/omnibus-graylog2/blob/2.3/files/graylog-cookbooks/graylog/attributes/default.rb
Thanks for this.
I just need one more thing… please could you provide me with an example. I have attempted to look at the ruby files (temlates, defaults etc).
I’m not sure where to put this config, or how to set it up…
I apologise for not knowing how to do this.
So I managed to turn off default discovery:
My config contains:
"custom_attributes": {
"graylog-server": {
"elasticsearch_discovery_enabled": false,
}
}
Then I poked around for a way to customise the elasticsearch_hosts property but concluded that it is not possible to customise it to what I need as I need the whole url customised.
AWS elastic search VPC secure elastic search has a url like:
https://vpc-graylog-klasjdkljsdflkjsdfklji.eu-west-1.es.amazonaws.com
I ended up editing my graylog.conf.erb I think it was to “hard code” the elasticsearch_hosts to this value.
And I logged a feature request to ask for the ability to fully customise the ealstics search endpoint…
Overriding the elasticsearch_hosts configuration setting is currently not possible when using the Graylog omnibus package (which is being used in the OVAs and AMIs).
If you feel this is a crucial feature, please file an issue at Pull requests · Graylog2/omnibus-graylog2 · GitHub
This being said, the omnibus package (AMI, OVA) is supposed to be a turnkey solution and was not designed for maximum customization. I’d recommend starting with your own Graylog setup instead of using the pre-baked AMI if you want to use the AWS Elasticsearch Service.
Thanks @jochen I ended up doing what you suggested (logged a feature request). See post before yours.
If I was more skilled in ruby et all I’d probably do a pull request as well 
One comment I do want to make about
This being said, the omnibus package (AMI, OVA) is supposed to be a turnkey solution and was not designed for maximum customization
The thing is it’s got a lot of customisation already. Like being able to turn off elasticsearch and customise the elasticsearch nodes etc such that I don’t think the addition of this extra piece of customisation would significantly affect it’s turnkey aspect.
Let me say that there is no way I would have got up running this quickly without this prebuilt AMI to start with.
Correct, but these are extremely opinionated and one opinion is that you should run Elasticsearch yourself and not use an external service.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.