Graylog does not start after enabling HTTPS support

Graylog Central (peer support)
1

Hi, there

In order to connect graylog with external elasticsearch via https I have enabled HTTPS support in conf file, generate self-signed certificate and add the certificate in JVM trust store .
But after that graylog service is not working. If http_enable_tls is disabled, everything is OK

Default: false
http_enable_tls = true

The X.509 certificate chain file in PEM format to use for securing the HTTP interface.
http_tls_cert_file = /root/graylog_cert/new/server.crt

The PKCS#8 private key file in PEM format to use for securing the HTTP interface.
http_tls_key_file = /root/graylog_cert/new/server.pem.org

The password to unlock the private key used for securing the HTTP interface.
http_tls_key_password = secret

This is the only messages that I got after service graylog-server start

root@PC156:~/greylog_cert/new# sudo tail -f /var/log/graylog-server/server.log
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
at com.github.joschi.jadconfig.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:53) ~[graylog.jar:?]
at com.github.joschi.jadconfig.JadConfig.invokeValidatorMethods(JadConfig.java:221) ~[graylog.jar:?]
at com.github.joschi.jadconfig.JadConfig.process(JadConfig.java:100) ~[graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.processConfiguration(CmdLineTool.java:351) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.readConfiguration(CmdLineTool.java:344) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:178) [graylog.jar:?]
at org.graylog2.bootstrap.Main.main(Main.java:50) [graylog.jar:?]

2

I found that there is an issue with private key file:

2020-03-09T14:50:28.973+02:00 ERROR [CmdLineTool] Invalid configuration
com.github.joschi.jadconfig.ValidationException: Unreadable or missing HTTP private key: /root/graylog_cert/pkcs8-encrypted.pem

Can you advise me how to generate proper pems file, I following the steps from https://docs.graylog.org/en/3.2/pages/configuration/https.html

3

he @alex1

when you enable tls you need to adjust http_publish_uri and http_external_uri to include https - the default is http…

if you use self-signed certificate, consider to use shadowCA (or similar helpers) to create the certificates.

4

Alex, try making the key file encrypted and don’t use the password in the config file. I have found that there is a bug that prevents the secret key from working and others on the community forums have found this also. There is a open bug report for this.

5

Thanks guys,

I followed all recommended steps from both of you, but i still got PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

root@PC156:~/graylog_cert/shadowCA/bin# ./create_ca_certificate.sh
Config file is unclean, cleaning it…

Generate Private Key …
choose a strong CA Password if requested!!

Generating RSA private key, 2048 bit long modulus
…+++
…+++
e is 65537 (0x010001)
Enter pass phrase for /root/graylog_cert/shadowCA/cert/CA/shadowCA.key:
Verifying - Enter pass phrase for /root/graylog_cert/shadowCA/cert/CA/shadowCA.key:

Sign ROOT Certificate with your previous entered CA Password …

Enter pass phrase for /root/graylog_cert/shadowCA/cert/CA/shadowCA.key:

Create DER from /root/graylog_cert/shadowCA/cert/CA/shadowCA.pem
Success
now you can Import
/root/graylog_cert/shadowCA/cert/CA/shadowCA.pem or
/root/graylog_cert/shadowCA/cert/CA/shadowCA.der
into your Systems Truststore

This is needed on all System that should trust this CA.
root@PC156:~/graylog_cert/shadowCA/bin# vi /root/graylog_cert/shadowCA/cert/CA/shadowCA.pem
root@PC156:~/graylog_cert/shadowCA/bin# vi /root/graylog_cert/shadowCA/cert/CA/shadowCA.key
root@PC156:~/graylog_cert/shadowCA/bin# cd …/cert/CA

root@PC156:~/graylog_cert/shadowCA/cert/CA# cp shadowCA.key shadowCA.pem /etc/graylog/server/cert/
root@PC156:~/graylog_cert/shadowCA/cert/CA# ls -l /etc/graylog/server/cert/
total 8
-rw------- 1 root root 1743 Mar 9 15:55 shadowCA.key
-rw-rw-rw- 1 root root 1371 Mar 9 15:55 shadowCA.pem
root@PC156:~/graylog_cert/shadowCA/cert/CA# chmod 777 /etc/graylog/server/cert/*
root@PC156:~/graylog_cert/shadowCA/cert/CA# ls
shadowCA.cnf shadowCA.der shadowCA.key shadowCA.pem
root@PC156:~/graylog_cert/shadowCA/cert/CA# ls -l /etc/graylog/server/cert/
total 8
-rwxrwxrwx 1 root root 1743 Mar 9 15:55 shadowCA.key
-rwxrwxrwx 1 root root 1371 Mar 9 15:55 shadowCA.pem
root@PC156:~/graylog_cert/shadowCA/cert/CA# cd /etc/graylog/server/cert/
root@PC156:/etc/graylog/server/cert# ls
shadowCA.key shadowCA.pem
root@PC156:/etc/graylog/server/cert# cp shadowCA.key shadowCA.key.pass
root@PC156:/etc/graylog/server/cert# openssl rsa -in shadowCA.key.pass -out shadowCA.key
Enter pass phrase for shadowCA.key.pass:
writing RSA key
root@PC156:/etc/graylog/server/cert# openssl rsa -in shadowCA.key -check
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA0uipj6z2zsTxmni1uTaZyuexdvUPWx65vkV+qOxpmVXovbzO
czlL+5tu58DuhQ990vovAWp6LN02LSTRqg+qgX2CgXBAzx0nT9SNBAor+NfdF6m5
VugcS7Cf3GduyAxBxDwfSHIse9nmfFxz0hQK9lPYXNypjyYQzhoBbJwDQ8MergUW
6b1+WYPuJ0bZpxtCqw8vXhT3Gf5W7L0I1TSv6MddhBZp5npKNGrQuKnxYpIYOmM5
3Qxn8itZ7ojU9xtLlRFHduidv2OjttKe1iOaQz38e1t0EDVqazzWEEOPFKCsYVNH
Tmll5j5Qwewl1jCkcpwT9xlVMc60vnoQUFsekQIDAQABAoIBAQCtLQUtR+RP2z5W
jA8KyV52UNBS0N2PACebqAxcP+JlWTkFsmYCDmXpbrMd8DjpYNL7zRmTnA49kFXP
s4/Eh6CFOr1R8w4EAfgpgGlZi9CXPwf3ShE43tV9AMYCf5ti/01xB7zYC4ejGkQ6
kN84++QUZrwdY8KGG8dos8m/mErF8435qD2QrS1IB5WXUVmp5JkQ4+dUjQdqNxkT
93DfYSulzxDg2qVC0x/USeZI7dFEeypm95HXSUQCVes189mDxzyIKivQVSPd9wj9
IBq+o1j7riaTSVXAeSj9UBGLHsPRphIyEQ6pzU50qegUUgrph393tz1tMSHJNxjS
8c4iOFLBAoGBAPw6EAPEICz9UtBVSSy8wO7xAQR64y68BGy1DkAZo24y5SF6bxRX
91x430y8c/I3uqbVT5VtzNOSwk78/fDYodc2ahbFYm4yU1AQvXUMP8/0qwTiJhmW
59eK/BCwx122V60Oh0xsrJovG//tJxJROd/pCmOiDQwBsTHMnCgJKRP9AoGBANYQ
Xe3wweH1FmRKaa47aBIzvhdv/WoZl6HkVROPqESDO+zqJL7lOclom4HDrLWlhVZT
PKpbq+qO0mtO3cQOfH9tthv1b1SpiWVE4FHZ4ifSNAnqsSItRHiHHXosY+XIRNSo
xjrObEgtE0xUY3+DICupEY9Gw4+kV2095SIgdZclAoGABhek8WJYYJ3R1URqGxGI
G31ox+HsAOH7vWB2Mqgr4WqLTE63xicFy48D3cejiVM9R/BZFsPrMaUW2m+T9rBU
4GzpDRnBceX3SzTMTrp4BB7lzp1jzDRv7y3lPsX2WtYfsgdqzzbEyIigxYLF8dRS
cI0hehyc8hBMZmaJ0xG9k0UCgYBpOwiBxL367zm/uZj0Dr4dXJRQ/zurYcNZ0V5F
THSioYTe9Ud1FEhxC1yO0PthHGpMwoRrFr+PggBMwitgiuKG3g8M9X1yOBmTz+Ua
N6rmen4cdeP38hrbyUgjXBkLx4SWqhUwqXMvlAENn9p9v58b93PrZBUU0uwrhqbS
PPSxhQKBgCyZOUG65WiU40KHMPkd0vtmQzRnUJKbEO8ECouRmAL75SnLulYtXlau
UjmV81omOmVrVAfo3vkkfcXM1drmZMEh8vOnM1Pm26jZMLtFPmy7v1T9frOJ14t1
19DVOBrii6u1s0NOf0LG5aOudLp2ReYUr7eAV504EdkYxKU/bhrs
-----END RSA PRIVATE KEY-----
root@PC156:/etc/graylog/server/cert# openssl x509 -in shadowCA.pem -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
c1:1c:b2:54:27:b4:f8:78
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = BG, ST = BG, L = Sofia, O = shadowCA, OU = Support shadowCA, CN = shadowCA (by jalogisch), emailAddress = a.altanbashev@dopamine.bg
Validity
Not Before: Mar 9 13:53:02 2020 GMT
Not After : Mar 8 13:53:02 2025 GMT
Subject: C = BG, ST = BG, L = Sofia, O = shadowCA, OU = Support shadowCA, CN = shadowCA (by jalogisch), emailAddress = a.t@test.bg
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:d2:e8:a9:8f:ac:f6:ce:c4:f1:9a:78:b5:b9:36:
99:ca:e7:b1:76:f5:0f:5b:1e:b9:be:45:7e:a8:ec:
69:99:55:e8:bd:bc:ce:73:39:4b:fb:9b:6e:e7:c0:
ee:85:0f:7d:d2:fa:2f:01:6a:7a:2c:dd:36:2d:24:
d1:aa:0f:aa:81:7d:82:81:70:40:cf:1d:27:4f:d4:
8d:04:0a:2b:f8:d7:dd:17:a9:b9:56:e8:1c:4b:b0:
9f:dc:67:6e:c8:0c:41:c4:3c:1f:48:72:2c:7b:d9:
e6:7c:5c:73:d2:14:0a:f6:53:d8:5c:dc:a9:8f:26:
10:ce:1a:01:6c:9c:03:43:c3:1e:ae:05:16:e9:bd:
7e:59:83:ee:27:46:d9:a7:1b:42:ab:0f:2f:5e:14:
f7:19:fe:56:ec:bd:08:d5:34:af:e8:c7:5d:84:16:
69:e6:7a:4a:34:6a:d0:b8:a9:f1:62:92:18:3a:63:
39:dd:0c:67:f2:2b:59:ee:88:d4:f7:1b:4b:95:11:
47:76:e8:9d:bf:63:a3:b6:d2:9e:d6:23:9a:43:3d:
fc:7b:5b:74:10:35:6a:6b:3c:d6:10:43:8f:14:a0:
ac:61:53:47:4e:69:65:e6:3e:50:c1:ec:25:d6:30:
a4:72:9c:13:f7:19:55:31:ce:b4:be:7a:10:50:5b:
1e:91
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
c8:37:8e:2b:89:16:f6:2c:4f:f1:90:75:6d:ac:3c:9c:32:28:
13:10:3c:0a:b8:36:65:3a:0a:39:f6:e9:5d:d7:46:43:f6:4f:
89:c9:b7:76:6e:30:a3:2e:e7:e5:d1:80:b9:91:4e:8d:44:9a:
6c:52:d2:74:74:ef:cb:88:fc:e0:6e:70:82:c2:69:99:03:a3:
40:5e:dd:65:db:bb:bc:ef:7c:a6:d6:57:65:a0:7a:21:ed:f3:
78:bf:c8:d3:a4:6f:9d:48:f7:2a:70:11:e2:53:da:9e:bb:dc:
ff:0e:c1:ff:1e:94:04:02:b1:25:b0:26:b6:8b:3f:16:5f:ce:
c4:f2:a1:a2:fc:df:bb:5b:21:e6:3a:85:ad:c6:3f:3c:7f:e4:
86:7b:8b:b2:1e:6d:c0:4c:d5:68:c8:71:3f:75:39:be:b1:ab:
31:e0:96:d5:91:9e:95:47:c2:15:ce:59:c6:fd:b4:17:08:48:
c8:a3:ea:08:70:3b:13:db:b9:7e:f3:e9:bc:28:8e:99:ed:c5:
77:65:57:b1:9d:84:b3:25:e1:cc:4b:29:2c:0e:7c:33:4f:cf:
ad:5b:a3:9d:c0:7c:04:74:3e:1a:86:9e:5a:ec:dd:82:4f:2f:
07:be:1f:4d:a0:db:83:ce:57:3f:07:72:3f:08:af:5b:fc:f6:
a2:5d:87:77
root@PC156:/etc/graylog/server/cert#

root@PC156:/etc/graylog/server/cert# keytool -importcert -keystore /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/cacerts -storepass changeit -alias graylog-self-signed -file /etc/graylog/server/cert/shadowCA.pem
Owner: EMAILADDRESS=a.altanbashev@dopamine.bg, CN=shadowCA (by jalogisch), OU=Support shadowCA, O=shadowCA, L=Sofia, ST=BG, C=BG
Issuer: EMAILADDRESS=a.altanbashev@dopamine.bg, CN=shadowCA (by jalogisch), OU=Support shadowCA, O=shadowCA, L=Sofia, ST=BG, C=BG
Serial number: c11cb25427b4f878
Valid from: Mon Mar 09 15:53:02 EET 2020 until: Sat Mar 08 15:53:02 EET 2025
Certificate fingerprints:
MD5: AA:04:25:ED:BE:B1:52:6D:FC:2A:6B:0A:4C:E8:D8:82
SHA1: 66:DF:84:51:A5:43:13:50:C9:A5:8A:CF:2F:A6:85:05:54:4B:06:E8
SHA256: FB:60:22:2C:53:72:14:60:66:D8:90:35:65:ED:AE:4F:F5:82:AB:50:8A:65:0F:4D:71:67:E7:5C:FE:63:93:4F
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 1
Trust this certificate? [no]: yes
Certificate was added to keystore
root@PC156:/etc/graylog/server/cert#

root@PC156:/etc/graylog/server/cert# cp shadowCA.pem /usr/local/share/ca-certificates/
root@PC156:/etc/graylog/server/cert# update-ca-certificates
Updating certificates in /etc/ssl/certs…
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d…

done.
done.
root@PC156:/etc/graylog/server/cert#

6

I have added certificate of external elastic in keystore and now everything is OK
Thanks a lot!

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.