Hey,
I am a little bit lost with how the TLS authentication should work with inputs. I have read the docs over and over, multiple posts about similar problems and I haven’t found any answers to my question yet so I decided to post one myself.
So what have I done:
- made all the necessary certificates and even changed the key format to pkcs8.
- tested the connection without the TLS and confirmed that Filebeat could send logs to Graylog
- added the CA.crt to the jvm.keystore as guided in the documents
- added the client.crt to the “trusted_clients” directory
- shared the certificates to correct locations and changed the permissions correctly
- tested the certificates with keytool and curl (connections OK)
- restarted everything
- configured the input to use TLS with authentication enabled
Beats input was configured as follows:
"tls_client_auth_cert_file": "/etc/graylog/server/trusted_clients/",
"tls_cert_file": "/etc/graylog/server/certs/server.crt",
"port": 5044,
"tls_key_file": "/etc/graylog/server/certs/server.key",
"tls_enable": true,
"tls_key_password": "",
"tls_client_auth": "required"
Filebeat is configured as follows:
output.logstash:
hosts: ["only.graylog.server:5044"]
ssl.certificate_authorities: ["/etc/filebeat/certs/ca.crt"]
ssl.certificate: "/etc/filebeat/certs/client.crt"
ssl.key: "/etc/filebeat/certs/client.key"
What I expected:
- logs to be sent over TLS
What I got:
- Some error messages saying that the configured path to “trusted_clients” is a directory
Next I configured the “TLS Client Auth Trusted Certs(optional)” to point to a client.crt inside the directory. I got an error saying “javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE)”.
Then I accidentally stumbled to this post: TLS Client authentication - null cert chain
So I tried the same thing and finally got the TLS authentication working when I moved CA.crt to the “trusted_clients” directory and pointed the “TLS Client Auth Trusted Certs(optional)” to it. I thought this was weird but it was working - although there was a new error in the Graylog server.log saying that “Client auth configured, but no authorized certificates / certificate authorities configured for input”.
But why? Shouldn’t there be all of the “client.crt” -files under the “trusted_clients” directory? (BTW, got the “TLS Client Auth Trusted Certs(optional)” working as directory by pointing the path to “/etc/graylog/server/trusted_clients/*” <- see the asterisk).
What I did next I removed the CA.crt from the directory but it still works. I removed the client.crt and it is still working. I even changed the path in the input configuration to “/etc/graylog/server/untrusted_clients/” and it still works. I have restarted everything.
This post was a bit long and maybe a bit hard to read but I hope someone got the gist of it. 
So I think my question here is that how to do this correctly?
Graylog versions: 3.0.2, 3.1.4, 3.2.4
MongoDB vesion: 4.0.12
Elasticsearch version: 6.8.2
And it started working when I pointed to the CA.crt which is already added in the keystore. It did not work with the client.crt.