Setup Beats to ship logs using TLS encryption and authentication

Graylog Central (peer support)
1

Hi,

I’ve been having issues using the documentation to get filebeat to ship encrypted logs.

If possible I want to use a single input with optional TLS authentication enabled so I can encrypt some logs being shipped between 2 data-centres but not use authentication for traffic within the same location as the graylog server. The following configuration does not work for clients I allocated the TLS enabled configuration, it also breaks the input so no further logs are being digested.

I see errors in the graylog server that the client’s cert cannot be accessed:

2020-02-07T08:27:50.566+02:00 WARN  [ChannelInitializer] Failed to initialize a channel. Closing: [id: 0x9663e0e0, L:/10.2.37.241:5044 - R:/10.2.37.243:47202]
    java.nio.file.AccessDeniedException: /etc/graylog/server/trusted_clients/client.crt

The file and folder is owned by root and is readable by everyone.
-rw-r--r-- 1 root root 1757 Feb 7 08:00 /etc/graylog/server/trusted_clients/client.crt

Further here is my input config:

image
image564×508 30.7 KB

And here is my filebeat config:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.source: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/httpd/*.log
    - /var/log/*.log
  type: log
output.logstash:
    hosts: ["graylog-server:5044"]
    ssl.certificate_authorities: ["/etc/ca.pem"]
    ssl.certificate: "/etc/graylog/sidecar/cert/${sidecar.nodeName}.crt"
    ssl.key: "/etc/graylog/sidecar/cert/${sidecar.nodeName}.key"
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

It would be appreciated if anyone can help. I’ve been stuck with this for a while now.

2

the @daniejstriata

if I remember correct it might be a bug in Graylog - as I can’t find the one, you might want to open one:

The problem you have should be solved if you refer ONE certificate file for the client auth trust certs and concat all certificates together. The reading of the directory is the problem.

3

Hi Jan, Thanks for your reply.

I’m not sure how a working configuration would look like at this stage. I’m creating a new set of certificates using my own CA process rather than ShadowCA. Do I need to install the CA cert for the OS and import it into Java?

I’m also assuming that the certificate used for the Beat input do not need to be imported into JVM as I am configuring the CA Cert from the graylog frontend. The cert I’m using for the Graylog frontend is issued by Let’s Encrypt and won’t match the internal hostnames in any way. I only need to add encryption to the beats input.

I’m still getting this error after concatenating them into one file:

2020-02-10T14:05:29.561+02:00 WARN [ChannelInitializer] Failed to initialize a channel. Closing: [id: 0xab3312c4, L:/10.2.37.241:5044 - R:/10.2.37.243:42308]
java.nio.file.AccessDeniedException: /etc/graylog/server/trusted_clients/client.crt

Consisting of:
Cert
CA
Key

I removed the entry and only left TLS certs for the server in place. That caused this to happen:

2020-02-10T14:13:20.989+02:00 WARN [AbstractTcpTransport] Client auth configured, but no authorized certificates / certificate authorities configured for input [Beats/5e17053d6af7041368136470]
2020-02-10T14:13:21.028+02:00 ERROR [AbstractTcpTransport] Error in Input [Beats/5e17053d6af7041368136470] (channel [id: 0x96bbc826, L:/10.2.37.241:5044 ! R:/10.2.37.166:58946]) (cause io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 3257000000033243000001dd785ebc94c18b9b4018c56d4aff8142eff2ce6a46635c33a7bdb4879e0a5b7a6829611c3f8d303a3233915dc4ffbd9812ca6e926eb790bdfae938efbddff792cf9ee7bdf1bcc58711b7ae69c93ad1f6e04858c242968431fb1a279ce59c6511cbf2ef0870db9213a570027c4441c281a36a1415241c02b8879ec0516a89000319dbe80e1c59944731a60056ef8d2470d4a650e1b0632c5108a0740d3ea26a14818fe885db81633908b354ba5e96825add454ad798a600b54ab656ef8da4add44a9174da80236515b1955887d5e62609d3759185c53a2dc2f5aadc64e52aaf6ec40a017aa36d4ff2f0cd78bcf0e1e8004dd7efdde963a99522e9b4d976baa46d539e0838c8e7233ad1d2c970a7ad3b3b184e0d9adfbd70d0144057952507beceb364930768c95a51cf7ffc44851f333f4e0f7131bffee3af6f1faca3b6fc11ffe4fe9d13c6355ded7f31fabed5f7feb78fbea15e3552b84677bed9771d99288a3025331e0bcf5bbc7b353ce618f8f8af1cbc5c7d3f90359125333492b87fb79792a8a432c235a93ac2f134eee045b83c8e7f95b38bdbf4df14ff0dbedf38bcf5bcc5fb57c3e168dcd9e5992f7b76f0d4e6c7c6a56976d1a15a25dbeb14cbf30d32cb99d93f2369ba98f4b5f7e5d016543e5316987e050000fffffadcefa03257000000033243000001fc785ec4944f8bdb3010c5dd947e8142efe69d65afecc4f9a3d35eda434f0b5b7a682941b6c78e41b68ca49884e0ef5eec406071d2dd6e697bf2c18366e6bddfbcf8b3e7796f3c6ff6e1847b57d5649dac5b08c43ce6018f83887f8962c1378247215f45dfc0705f9393b97412e28494a4834051294a493a30b8634b10c87506868e8cad74038165b80e23f40cba282c398864bd4ce284a1266b654910f844a91f713f5a08be119cfba54955d0ed388f956f8fd6519d7f8f7e08ffd149e3aaa6f41f8c3ed4fae07ffde81b6a55954957e9c637fba62113862118aaa6dd3b8813dcb12508285da26748493a881376daba46d60481d2a42ae8769cc70ad3c1199a6959cf60f5de64c3c3a5b90c0b06a54b88138a4ad1f06da5db41e0ae93e64ee9f22e9754eb261caafa9e21d34a51e6b4d9363aa76d954fde2b55bc3db7da5e8a21b0e005f1b94c8262b38a8345922e833459a44132cf37cb7cbe2e56720e86d668db8e0d8661dc7134686cceb0d376d4e7da7afd88c7ccf366effe191ee72d2702e8e242cd92af5e434ddb9135a125d3551909ff719f654439e5e15f957750f9e520544dbb1fdd78ead1c5f29b7c8cb777ddc5b3c557ec9d42fe221cde7adeecfdffc6e1f764adc9be266228f71f8c3edc4e98e7f3e5aaecc3c95dfdd14dd372a8bd616cfff42a5673f6cb437f1ea23f0999fe67000000ffff667aef823257000000033243000001e3785ebc)

4

Hi @jan
Have you been able to look into my question yet? I’m not making any progress.

5

Is the issue that I don’t use encryption on the graylog server service? Should I use TLS for the server so that I can use TLS for the beat input?

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.