Problem with enabling https, using Let's Encrypt Wildcard SSL certs

Graylog Central (peer support)
1

Hello,
I’m having problem with SSL configuration usingg Let’s Encrypt Wildcard certificates.
I’ve created /etc/certs/ directory and put all required LE certificates there. Directory is owned by graylog user/group, having full permissions to them.

Environment is “fresh” I mean it was recently installed on Debian 11, with Elasticsearch 7.16.2, mongodb 4.4.10 and openJDK 11.0.13

Here is some config from graylogs server.conf

http_bind_address = <graylog_ip_addr>:9000
http_publish_uri = https://<graylog_domain_name>:9000/

http_enable_tls = true
http_tls_cert_file = /etc/certs/cert.pem
http_tls_key_file = /etc/certs/privkey.pem

When accessing https://<graylog_domain_name>:9000 - GUI works, but for example when I go to the System/Inputs - it shows that none input is working (which is not true, because I can refresh search and I see new logs are comming into graylog).
Also when I access System/Nodes, and click on the node I’ve got 2 error messages:
Could not get plugins
Getting plugins on node “80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08” failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Failed to connect to <graylog_domain_name>/127.0.1.1:9000

Could not get JVM information
Getting JVM information for node ‘80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08’ failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Failed to connect to <graylog_domain_name>/127.0.1.1:9000

Of course I’ve imported keys to java with this command:

sudo keytool -importcert -keystore /etc/certs/graylog_keystore.jks -storepass <some_password> -alias <graylog_domain_name> -file /etc/certs/cert.pem

And also edited the /etc/default/graylog-server, added this line at the end of 1st line of Default java options

-Djavax.net.ssl.trustStore=/etc/certs/graylog_keystore.jks -Djavax.net.ssl.trustStorePassword=<some_password>

When I try to curl by domain name it just throw connection refused:

curl -i https://<graylog_domain_name>:9000/api/?pretty=true
curl: (7) Failed to connect to <graylog_domain_name> port 9000: Connection refused

Do I missing something?

2

Hello,

Maybe I can help. There could be a couple reason for this.

  1. Are you accually using a loop back address ( 127.0.0.1.)? If so make sure your /etc/hosts/ file has something like this configured. Yours maybe different since your using a loop back address.
    Example.
[root@graylog graylog_user]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.34  graylog.domain.com

To be honest I would have used a default IP Address and not my 127.0.0.1, then I would have made it static address.

  1. If you are actually using a IP address other then 127.0.0.1 make sure your Hostname is correct also make sure your /etc/hosts file is correct, and if this is in a domain check your DNS server has a reverse lookup for that Address.

  2. Last, in your Certificates check if you have an subjectAltName.

subjectAltName = @alt_names
# IP addresses and DNS names the certificate should include
# Use IP.### for IP addresses and DNS.### for DNS names,
# with "###" being a consecutive number.
[alt_names]
IP.1 = 203.0.113.42
DNS.1 = graylog.example.com

Hope that helps

EDIT: I forgot do you have a firewall enabled?
Also

This is not the correct version shown in the documentation but if its working you may have some issues later.

3

Hello,
I’ve checked the hosts file, it was like:

127.0.0.1      localhost
127.0.1.1       graylog.domain.com    graylog

I’ve changed it to (but also tried graylog interface ip addr):

127.0.0.1       graylog.domain.com    graylog localhost

Still not working, but getting different errors:

Getting JVM information for node '80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08' failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Hostname <graylog_ip_address> not verified: certificate: sha256/cjDEtdwMnpCYHREA+mPg6WgjJxk1sjNS65Veu38nO+k= DN: CN=*.domain.com subjectAltNames: [*.domain.com]

Getting plugins on node "80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08" failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Hostname <graylog_ip_address> not verified: certificate: sha256/cjDEtdwMnpCYHREA+mPg6WgjJxk1sjNS65Veu38nO+k= DN: CN=*.domain.com subjectAltNames: [*.domain.com]

When viewing Lest’s Encrypt Wildcard cert.pem file in openssl I have this:

X509v3 Subject Alternative Name:
                DNS:*.domain.com

Ofc all pasted domain names here were replaced, but they’re matching in all places.

Firewall is disabled.

4

Hello

Getting JVM information for node ‘80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08’ failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Hostname <graylog_ip_address> not verified: certificate: sha256/cjDEtdwMnpCYHREA+mPg6WgjJxk1sjNS65Veu38nO+k= DN: CN=.domain.com subjectAltNames: [.domain.com]

Getting plugins on node “80bd2cdb-c1a8-42ab-a9e5-7272cbe9df08” failed: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Hostname <graylog_ip_address> not verified: certificate: sha256/cjDEtdwMnpCYHREA+mPg6WgjJxk1sjNS65Veu38nO+k= DN: CN=.domain.com subjectAltNames: [.domain.com]

Something is wrong with you certificate configuration. As shown above in the error log.

 Hostname <graylog_ip_address> not verified: certificate

127.0.0.1 <— This is your home address AKA loop back. Does your server have an IP address beside 127.0.0.1. If so that should be there as I showed above.

192.168.1.34 graylog.domain.com

The reason I say this is because of this error above. You have Graylog FQDN connecting to a home address and not its public/private IP Address as shown below.

You may want to check out this documentation. These docs has all the information needed for making sure your certificate are correct for Graylog.

https://docs.graylog.org/v1/docs/sec-adcs-certificates

https://docs.graylog.org/v1/docs/https

If that doesn’t work can you show us what you did? If by chance you have a DNS server in your environment make sure Graylog server has a PTR (reverse lookup) configured.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.