Hello there!
I saw many similar topics,
but, I believe here’s something different:
I’m trying to setup SSL (LetsEncrypt suff) for my Graylog setup, using Docker Compose.
I’m stuck with the following error:
( “https+://” - I’m new user. 2 links limit. )
graylog_1 | 2020-12-01 19:20:31,382 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call https+://172.24.0.4:9000/api/system/metrics/multiple on node <b9d3468d-d294-439e-9e04-747876c2e448>: Hostname 172.24.0.4 not verified:
I’m setting HTTPS for my domain - mydomain.example.com.
And what seems most strange to me, - I can go to https+://mydomain.example.com:9000/api/system/metrics/multiple
yes, on port 9000 with, yes, all the LetsEncrypt SSL stuff.
So SSL is working for me!
Good.
But I cannot make Graylog go to API by mydomain.example.com.
It always goes to 172.24.0.4!
That’s really frustrating.
I’ve lost whole day, trying to configure this address.
Looks like played with all the “puzzles” with GRAYLOG_WEB_ENDPOINT_URI, GRAYLOG_REST_TRANSPORT_URI, GRAYLOG_HTTP_EXTERNAL_URI, GRAYLOG_REST_LISTEN_URI etc.
Here’s my actual docker-compose.yml:
( “https+://” - I’m new user. 2 links limit. )
When you say “make Graylog go to API by mydomain.example.com”, what do you mean? How are you directing Graylog to this URL?
The Graylog JVM trust store includes trust for major root CAs by default I believe, but I don’t know if that includes LetsEncrypt. You may need to add the certificate chain to the trust store for this to work for you.
As I wrote above - all the ssl-stuff already added.
I followed tons of instructions from the i-net.
But IP address which is used to access “backend” REST API is not related with certificate in any way.
The main problem is - whatever I did, GrayLog always tries to use internal (Docker network?) IP (172.24.0.4) -
Unable to call https://172.24.0.4:9000
I want to configure GrayLog, so it use https://mydomain.example.com:9000/api… for REST calls.
Because I can access this SSL-ed address from browser.
But, I cannot tell GrayLog to use this DNS. It always goes to internal IP.
#### HTTP bind address
#
# The network interface used by the Graylog HTTP interface.
#
# This network interface must be accessible by all Graylog nodes in the cluster and by all clients
# using the Graylog web interface.
#
# If the port is omitted, Graylog will use port 9000 by default.
#
# Default: 127.0.0.1:9000
http_bind_address = 10.1.1.100:9000
#http_bind_address = [2001:db8::1]:9000
According to my docker-compose shown above - no. (this time)
Which value should I use for http_bind_address (GRAYLOG_HTTP_BIND_ADDRESS) for this HTTPS case?
GRAYLOG_HTTP_BIND_ADDRESS=https://mydomain.example.com:9000 ?
@jan thanks for answer.
As I wrote above - “I followed tons of instructions from the i-net” , including official documentation.
OK, now my config looks like this:
WHY Graylog keep trying this internal 172.24.0.4 address for REST API ?!
I want Graylog to use Letsencrypt-certified external address.
How to configure it?
This external address is up and running perfectly well.
Here’s my Graylog welcome screen, which is working and HTTPS-enabled:
Graylog will connect to itself for various reasons. This connection is done to the http_publish_uri- means your certificate need to include the hostname/ip that is configured for that
The HTTP URI of this Graylog node which is used to communicate with the other Graylog nodes in the cluster and by all clients using the Graylog web interface.
The URI will be published in the cluster discovery APIs, so that other Graylog nodes will be able to find and connect to this Graylog node.
This configuration setting has to be used if this Graylog node is available on another network interface than $http_bind_address, for example if the machine has multiple network interfaces or is behind a NAT gateway.
This configuration setting must not be configured to a wildcard address!
If http_bind_address contains a wildcard IPv4 address (0.0.0.0), http_publish_uri will be filled with the first non-loopback IPv4 address of this machine instead.
so you need to make that include the right protocol via configuration PLUS the certificate need to be valid for that configured ip/hostname.
Wooohoooo!!!
This was a missing part of my “puzzle”! (in particular combination!) - GRAYLOG_HTTP_PUBLISH_URI=https://mydomain.example.com:9000/
Thank you @jan so much for idea!
So, for everyone, - this is my working docker-compose.yml:
version: '3'
volumes:
local_storage_nosql:
services:
nosql:
image: mongo:4.0
volumes:
- local_storage_nosql:/data/db
- ./graylog.js:/docker-entrypoint-initdb.d/graylog.js:ro
network_mode: "host"
# Due to HOST network, open MONGODB only for local interface:
command: mongod --bind_ip 127.0.0.1
environment:
# provide your credentials here
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=passwordpasswordpassword
ports:
- 27017:27017
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
network_mode: "host"
# Due to HOST network, open ELASTICSEARCH only for local interface (127.0.0.1)
environment:
- http.host=127.0.0.1
- discovery.type=single-node
- xpack.security.enabled=false
- transport.host=localhost
- network.host=127.0.0.1
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
graylog:
image: graylog/graylog:4.0
# Using HOST network, for easier GrayLog self-requests. ATTN! Firewall rules!
network_mode: "host"
volumes:
- ./g.crt:/usr/share/graylog/data/config/ssl/cert.crt:ro
- ./g.key:/usr/share/graylog/data/config/ssl/key.key:ro
environment:
- GRAYLOG_ROOT_TIMEZONE=Europe/Riga
- GRAYLOG_IS_MASTER=true
# CHANGE ME!
- GRAYLOG_PASSWORD_SECRET=STlvln2h5I2OsF0Hz0ebJSZG9SzJ0NYpnQTyRh3VJ2aVQ95cWwcTXSzgMU18ryNSsG2n9Voj4q7dFZqhwrNC1gvZd3VDLTPY
- GRAYLOG_ROOT_PASSWORD_SHA2=9a3f982d63aee554970b640fcec5688378756e3e1eda0829495a7b59c395188b
- GRAYLOG_HTTP_ENABLE_GZIP=true
- GRAYLOG_HTTP_ENABLE_TLS=true
- GRAYLOG_HTTP_TLS_CERT_FILE=/usr/share/graylog/data/config/ssl/cert.crt
- GRAYLOG_HTTP_TLS_KEY_FILE=/usr/share/graylog/data/config/ssl/key.key
- GRAYLOG_MONGODB_URI=mongodb://graylog:passwordpasswordpassword@127.0.0.1:27017/graylog
- GRAYLOG_MESSAGE_JOURNAL_ENABLED=false
- GRAYLOG_HTTP_EXTERNAL_URI=https://mydomain.example.com:9000/
- GRAYLOG_HTTP_PUBLISH_URI=https://mydomain.example.com:9000/
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_ELASTICSEARCH_HOSTS=http://127.0.0.1:9200
- GRAYLOG_ELASTICSEARCH_VERSION=7
depends_on:
- nosql
- elasticsearch
ports:
- "9000:9000"
- "12201:12201/tcp"
- "12201:12201/udp"
And notice - there’s no any mentions of *.JKS (Java Key Storage) files, related Java settings, etc.
SSL/HTTPS is working with only 2 files - cert.crt and key.key which were received from LetsEncrypt CertBot for mydomain.example.com
just to be complete - you do not need a custom JKS, because you use lets-encrypt certificates that can be verified with the given CAs on a default debian system (what the image based on). If you had created your own self-signed certificates it would have been different.
In addition when http_external_uri and http_publish_uri are the same - only the later one is needed to be defined. As the default of http_external_uri is what is in http_publish_uri.
Plus this is all working in your environment because mydomain.example.com is also resolvable to the container IP … (or Graylog is able to reach itself over that domain name to be exact).
Yes, @jan good point.
And I’ve updated docker-compose configuration above:
Using Compose version 3
Using HOST network
Restricting MongoDB and ElasticSearch to local interface only (due to HOST network)
I’ve updated to Docker HOST network, because I’ve experienced some problems on other “clean” host,
GrayLog said:
WARN [ProxiedResource] Unable to call http://mydomain.example.com:9000/api/system/metrics/multiple on node <8sj4991c-ed04-4005-8ad5-c17fd7f443fd>
java.net.SocketTimeoutException: connect timed out
So, the HOST network for Docker solved this problem.
