After 3 days of fighting with certs to get my instance back up and running I’d like to share my thoughts about the process and what major suggestion I would add to the documentation.
First of all the suggestion. COMMON NAME: need to be more specific that the common name needs to be EXACTLY what you set in the rest_uri and web_uri otherwise you’ll get java errors like “javax.net.ssl.SSLPeerUnverifiedException: Hostname not verified:” or other similarly cryptic messages. I would have been done 3 days ago if that simple point was made and where to specify CN.
I have followed many suggestions on this forum and a few tips have led me to the conclusion that Common Name is way more important than all the other fields when setting up your certs but there is not enough of a stress on that point. Furthermore, all the help from devs points to the documentation or this script or this github repo for setting up certs. Only the github repo was helpful to create working certs and that is mainly what I used to do so, mixed with a tip or two from the official documentation mainly about the keystore. To summarize, this is what I needed to do to setup my system:
Copy keystore from your java location, on our system it’s like so:
> cp -a /usr/java/jdk1.8.0_161/jre/lib/security/cacerts cacerts.jks
Change or don’t change the default password “changeit”
Follow the steps from the github repo above. Making sure to put the FQDN or what you will put in for rest_uri and web_uri in your graylog server.conf file as Common Name. Eg:
> keytool -genkey -alias fqdn -keyalg RSA -validity 365 -keystore keystore.jks
What is your first and last name? [Unknown]: FQDN
and
> openssl req -x509 -days 365 -nodes -newkey rsa:2048 -keyout pkcs5-plain.pem -out cert.pem
Common Name (eg, your name or your server's hostname) []: FQDN
I then continued with the commands from the git repo but instead of modifying the default java keystore I added my certs to the local copy as suggested in the official documentation.
Modify permissions on all *.pem, *.jks, *.p12 for the graylog user. Copy them to a new certs directory or use that path when updating server.conf.
Next you can modify server.conf to point to the new cert/key directory and update the rest and web uri to the fqdn specified when creating the certs.
Finally, if you have made a cop of the java keystore, modify the startup script for graylog-server to include the path to your copied keystore. (eg, -Djavax.net.ssl.trustStore=/path/to/cacerts.jks) Without root permission on your system it may be needed to skip the step of making a copy of the default java keystore and modify that one in place. Then you would not need to add that line to the graylog-server startup options. I have not tested this theory, YMMV.
I believe that was all. Took me nearly 3 days to figure that out and I hope this will stop someone else from banging their head against the wall for something so simple and that one line in the documentation would solve. I realize in the official docs that it is suggested to setup a .cnf file with your settings. This would mostly work except it’s not stressed that the CN needs to be exactly as it is in server.conf. Also the structure of the docs is not friendly for someone who knows nothing about certs and it goes off on tangents about converting other keys and converting an existing java keystore before getting back to the process of importing your new certs into a copied version of the default java keystore. The CN suggestion above and reorganizing that documentation would go a long way to help new users out.
Also, another commonly referenced help is the script mentioned above. While this is a great tool and I appreciate the effort going into it, it also has a serious flaw with regards to the CN. If you use “Magic Mode” it will set the CN to a shortened hostname, not FQDN so, unless your rest and web uri’s are solely hostname, this will not work. Using the -h flag to specify a FQDN will work better. The script does not ask or gather from a config file the rest of the (typically) required distinguished name details like OU, Locality or State etc which I believe don’t hamper the cert but it’s good to have.
All in all, this was a learning experience and I hope this helps someone else down the road. Thanks for reading.