1. Describe your incident:
Note that :// has been {obfuscated} to eliminate detection of extraneous links from config files and cer/key files.
Following the recipe described here: Enable HTTPS using existing certificates bought from GoDaddy - #2 by vneerukonda
We have used openssl to convert our wildcard key and cert and chain files to create a .pfx file, which we then used, following the above recipe, to create dedicated graylog.cer and graylog.key files. We ensured the files are owned by graylog:graylog in a sub-directory of /etc/graylog/ssl also owned by graylog:root
Used the recipe above, created the files, added the new cert to the cacerts.pks, updated the graylog java options file and still are getting the error:
com.github.joschi.jadconfig.ValidationException: Unreadable or missing HTTP X.509 certificate: null
2. Describe your environment:
-
OS Information: Ubuntu 22.04 LTS
-
Package Version: dpkg --list | grep graylog
ii graylog-5.0-repository 1-2 all Package to install Graylog 5.0 GPG key and repository
ii graylog-server 5.0.5-1 amd64 Graylog server -
Service logs, configurations, and environment variables:
Config:
/etc/graylog/server# grep -v “#” server.conf | grep -v ^$
is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret ={obfuscated}
root_username = admin
root_password_sha2 ={obfuscated}
root_email = “admin@{obfuscated}.{obfuscated}.org”
root_timezone = US/Central
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = gl5log01.{obfuscated}.com:9000
http_publish_uri = https{obfuscated}gl5log01.{obfuscated}.com:9000/
http_external_uri = https{obfuscated}gl5log01.{obfuscated}.com:9000/
http_enable_cors = true
http_enable_tls = true
http_tls_cert = /etc/graylog/ssl/graylog.cer
http_tls_key_file = /etc/graylog/ssl/graylog.key
stream_aware_field_types=false
elasticsearch_hosts = http{obfuscated}opensearch00:9200,http{obfuscated}opensearch01:9200,http{obfuscated}opensearch02:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
message_journal_max_size = 6gb
lb_recognition_period_seconds = 3
mongodb_uri = mongodb{obfuscated}glogdb01.{obfuscated}.com:27017,glogdb02.{obfuscated}.com:27017/graylog?replicaSet=MongoCluster
mongodb_max_connections = 1000
mongodb_version_probe_attempts = 5
transport_email_enabled = true
transport_email_hostname = smtp01.{obfuscated}.com
transport_email_port = 25
transport_email_use_auth = false
transport_email_from_email = noreply@{obfuscated}.com
enabled_tls_protocols = TLSv1.2,TLSv1.3
skip_preflight_checks = false
Folder where cer and key reside:
/etc/graylog/ssl# ll
total 28
drwxr-xr-x 3 graylog root 4096 Mar 31 13:09 ./
drwxr-xr-x 4 root root 4096 Mar 30 10:46 …/
lrwxrwxrwx 1 root root 27 Jan 24 13:02 cacerts.jks → /etc/ssl/certs/java/cacerts
-rw------- 1 graylog graylog 2594 Mar 31 12:22 graylog.cer
-rw------- 1 graylog graylog 1832 Mar 31 12:23 graylog.key
drwxr-xr-x 4 root root 4096 Mar 31 13:09 old/
-rw------- 1 root root 6963 Mar 31 12:21 wildcard.{obfuscated}.com.pfx
Beginning of graylog.cer file:
:/etc/graylog/ssl# cat graylog.cer
Bag Attributes
localKeyID: AB FB 93 FD 5E 91 53 B3 78 06 37 F1 7A 0B 12 96 C8 CC 3A 08
subject=CN = *.gvtc.com
issuer=C = US, ST = Arizona, L = Scottsdale, O = “GoDaddy {dot}com, Inc.”, OU = http{obfuscated}certs.godaddy {dot} com/repository/, CN = Go Daddy Secure Certificate Authority - G2
-----BEGIN CERTIFICATE-----
MIIGgTCCBWmgAwIBAgIITRwrDwj+cqcwDQYJKoZIhvcNAQELBQAwgbQxCzAJBgNV
BAYTAlVTMRAwDgYDVQQIEwdBcml6b25hMRMwEQYDVQQHEwpTY290dHNkYWxlMRow
{remainder deleted}
Contents of graylog.key file:
/etc/graylog/ssl# cat graylog.key
Bag Attributes
localKeyID: AB FB 93 FD 5E 91 53 B3 78 06 37 F1 7A 0B 12 96 C8 CC 3A 08
Key Attributes:
-----BEGIN PRIVATE KEY-----
MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDd3mliS0Vs+Pv0
4ZT38WUYcGdM8p7HymuN3xkTCXr2fQcZUzggFA5bm4xsucqWUCZK28qRo0qaqiEw
{remainder deleted}
3. What steps have you already taken to try and solve the problem?
Multiple paths and methods of converting apache, tomcat, iis versions of GoDaddy certificate files to formats that would be acceptable to Graylog.
Really not sure why the standard Tomcat version won’t simply work without conversions, but no method of converting using openssl has worked for us so far.
4. How can the community help?
We’ve found many places where folks are using self signed certs, but only a couple references to using a wildcard public signed cert from a recognized CA authority and only the .pfx version, which my provider, GoDaddy doesn’t provide, is even mentioned for an effective recipe for conversion.
Identify a pathway to take a standard certificate/key file from a national provider and make it work.
It would also be nice if the documentation links from inside Graylog would work for older versions instead of all generating a 404 error, but that’s a different issue…
We have an existing graylog4 deployment and are trying to build a new graylog5 with new backend storage using latest OSes and packages, this time enabling security as it is intended to be used by external departments and are finding it exceedingly difficult to get this working. It’s extremely frustrating to be trying seemingly random openssl conversion routines when documentation could share a recipe for use of one of the standard 3 formats of fully signed public certificates (apache, tomcat or even, iis).
