Graylog 4.3 TLS/HTTPS Certificate Configuration

**1. Describe your incident:**
Unable to get certificates to work for web page HTTPS mode.   

**2. Describe your environment:** 
RHEL8 Linux. FIPS disabled.

* Package Version:
graylog-server-4.3.8.1
OpenSSL 1.1.1k  FIPS 25 Mar 2021

**3. What steps have you already taken to try and solve the problem?**
I have followed these instructions:   
https://docs.graylog.org/docs/https

**4. How can the community help?**
Verify my procedures, what am I missing????

====================================================================
openssl-graylog.cnf
[req]
default = 2048
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no

[req_distinguished_name]
C = AAA
ST = BBB
L = CCC
O = DDD
OU = EEE
CN = graylog

[v3_req]
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
IP.1    = 1.2.3.4
DNS.1   = graylog
DNS.2   = graylog.test
====================================================================
Steps I did:
mkdir /etc/graylog/certs/
cp -a /etc/pki/ca-trust/extracted/java/cacerts /etc/graylog/certs/cacerts.jks

openssl req -x509 -days 365 -nodes -newkey rsa:2048 -config openssl-graylog.cnf -keyout pkcs5-plain.pem -out cert.pem

openssl pkcs8 -in pkcs5-plain.pem -topk8 -nocrypt -out pkcs8-plain.pem

openssl pkcs8 -in pkcs5-plain.pem -topk8 -out pkcs8-encrypted.pem -passout pass:secret

keytool -import -file cert.pem -keystore cacerts.jks -alias graylog -storepass changeit

keytool -import -file graylog-certificate.pem -keystore cacerts.jks -alias graylog2 -storepass changeit
====================================================================
systemctl stop graylog-server

vi /etc/graylog/server/server/conf
http_enable_tls = true
http_tls_cert_file = /etc/graylog/certs/cert.pem
http_tls_key_file = /etc/graylog/certs/pkcs8-encrypted.pem
http_tls_key_password = secret

systemctl start graylog-server
====================================================================
Error:
2022-11-21T20:55:19.267Z ERROR [CmdLineTool] Invalid configuration
com.github.joschi.jadconfig.ValidationException: Unreadable or missing HTTP private key: /etc/graylog/certs/pkcs8-encrypted.pem
at org.graylog2.configuration.HttpConfiguration.validateTlsConfig(HttpConfiguration.java:255) ~[graylog.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at com.github.joschi.jadconfig.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:53) ~[graylog.jar:?]
at com.github.joschi.jadconfig.JadConfig.invokeValidatorMethods(JadConfig.java:221) ~[graylog.jar:?]
at com.github.joschi.jadconfig.JadConfig.process(JadConfig.java:100) ~[graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.processConfiguration(CmdLineTool.java:465) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:271) [graylog.jar:?]
at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
====================================================================
====================================================================
i also made a jks keystore and exported certs and keys and changed configuration to match these file names..

keytool -importkeystore -srckeystore graylog.jks -destkeystore graylog.p12 -deststoretype PKCS12
openssl pkcs12 -in graylog.p12 -nokeys -out graylog-certificate.pem
openssl pkcs12 -in graylog.p12 -nocerts -out graylog-pkcs5.pem
openssl pkcs8 -in graylog-pkcs5.pem -topk8 -out graylog-key.pem
====================================================================
I get same error no matter what private key I try to use.
permissions are root:graylog at /etc/graylog and below.....

tks

Hello @rgsurfs

If you could tighten this post up a bit, its kind of hard to read. Using the mark down for configuration files and commands lines would be appreciated.

I managed to look over this post a couple times, you left information out to help us, help you trouble shoot this issue.

I’m not seeing this part here from the documentation.

The working directory should now contain the PKCS#8 private key (graylog-key.pem) and the X.509 certificate (graylog-certificate.pem) to be used with Graylog:

Example:

http_enable_tls = true
http_tls_cert_file = /etc/ssl/certs/graylog/graylog-certificate.pem
http_tls_key_file = /etc/ssl/certs/graylog/graylog-key.pem

Sorry for formatting, cleaned it up…

I am Following these instructions:
https://docs.graylog.org/docs/https

Step 1 - Creating a self-signed private key/certificate
No issues. I created my certificate with the openssf-graylog.cnf file.

Step 2 - Create PKCS#5 private key and X.509 certificate:
openssl req -x509 -days 365 -nodes -newkey rsa:2048 -config openssl-graylog.cnf -keyout pkcs5-plain.pem -out cert.pem
No issues.

Step 3 - Convert PKCS#5 private key into a unencrypted PKCS#8 private key:
openssl pkcs8 -in pkcs5-plain.pem -topk8 -nocrypt -out pkcs8-plain.pem
No issues

Step 4 - Convert PKCS#5 private key into an encrypted PKCS#8 private key (using the passphrase secret):
openssl pkcs8 -in pkcs5-plain.pem -topk8 -out pkcs8-encrypted.pem -passout pass:secret
No issues

I now have cert.pem and pkcs8-encrypted.pem files for use.

The next step: Converting a PKCS #12 (PFX) file to private key and certificate pair/
I am not using a pfx keystore.

The next step: Converting an existing Java Keystore to private key/certificate pair
I am not using java keystores.

So I should be able to use the cert.pem and pkcs8-encrypted.pem files for use. correct ???

type or paste code here

hello,

I’ll try to make this simple for ya, It can be difficult to do. Replace IP address and FQDN you your environment. This is on CentOS 7

Change the default password for Java CAcert store. Default password is changeit

keytool -storepasswd -keystore  /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre/lib/security/cacerts

Type “changeit”
New-PASS Type “secret” /// this will be configured in graylog configuration file

Create a file named openssl-graylog.cnf with the following content.

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
# Details about the issuer of the certificate
[req_distinguished_name]
C = US
ST = iowa
L = cedar rapids
O = enseva
OU = admin
CN = graylog.domain.com
[v3_req]
keyUsage = keyEncipherment, dataEncipherment,nonRepudiation
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
# IP addresses and DNS names the certificate should include
# Use IP.### for IP addresses and DNS.### for DNS names,
# with "###" being a consecutive number.
[alt_names]
IP.1 = 10.10.10.10
DNS.1 = graylog.domain.com

I execute the following commands to create the certs for Graylog

mkdir /etc/ssl/certs/graylog/ && cd /etc/ssl/certs/graylog/
openssl req -x509 -days 1095 -nodes -newkey rsa:2048 -config openssl-graylog.cnf -keyout pkcs5-plain.pem -out cert.pem
openssl pkcs8 -in pkcs5-plain.pem -topk8 -nocrypt -out pkcs8-plain.pem
openssl pkcs8 -in pkcs5-plain.pem -topk8 -out pkcs8-encrypted.pem -passout pass:secret
openssl pkcs12 -in keystore.pfx -nokeys -out graylog-certificate.pem
openssl pkcs12 -in keystore.pfx -nocerts -out graylog-pkcs5.pem
openssl pkcs8 -in graylog-pkcs5.pem -topk8 -out graylog-key.pem

The resulting graylog-certificate.pem and graylog-key.pem can be used in the Graylog configuration file.

I used default store “cacerts” with no configuration with the following commands

keytool -importcert -keystore /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre/lib/security/cacerts -storepass changeit (secret) -alias graylog.domain.com -file cert.pem

Configure Graylog config file:

http_bind_address = graylog.domain.com:9000
http_publish_uri = https://graylog.domain.com:9000/
http_enable_tls = true
 
http_tls_key_password = secret
 
## I used the directory where i create the certs and chown graylog:graylog -R.
http_tls_key_file = /etc/ssl/certs/graylog/graylog-key.pem
 
http_tls_cert_file = /etc/ssl/certs/graylog/graylog-certificate.pem

NOTE: ensure Graylog can read the certificates, meaning permission.
Example:

chown graylog:graylog  /etc/ssl/certs/graylog/graylog-certificate.pem

As for my Inputs I put my certificates needed in Graylog’s home directory .,Since Graylog owns that directory

EDIT: For lab environment you can set the PASSPHRASE on the certs to “Secret”.

Once you get the hang of it you can always create/change it later on. The reason I suggested using JAVA default key store is to understand how it works first. Later on this can be adjusted but it should work, This is from my own personal documentation. Also note I do not have to adjust or configure JVM parameter because the above is default, only thing I suggested was changing the Java Default keystore password.

Hope that helps.

1 Like

Thanks for breaking that all down. I am up now.

1 Like

Awesome @rgsurfs

Glad you resolved you issue could you explain what was the issue and mark this post as resolved , that would be great for future searches. :+1:

FIX: chown -R graylog:graylog /etc/graylog/

I am up on the https webpage and now I am getting these errors and inputs won’t go green/start, but it looks like I am receiving data though…weird…

2022-11-25T17:14:25.273Z WARN [ProxiedResource] Unable to call https://10.232.2.76:9000/api/system/metrics/multiple on node <b5aa6
49c-ae8a-496f-856b-392eb6a8e3ed>: Hostname 10.232.2.76 not verified:
certificate: sha256/Q65iFcJr+UH1ashr3F9EvxdJoipCrVkjhGwJCt0KfvE=
DN: CN=10.232.2.76, OU=XXX, O=XXX, L=XXX, ST=XX, C=US
subjectAltNames:

and I just fixed that… I changed from ip to hostname in server.conf
http_bind_address = graylog:9000
http_publish_uri = https://graylog:9000/

tks for all the help…

1 Like