I am currently facing an issue in sending data through collector sidecar using beats with SSL setup.
Consider this as our scenario,
I have two instances in which first instance have only running collector sidecar and the second instance are running Graylog application with SSL setup.
I am trying to collect and send data from first instance collector_sidecar to my second instance graylog application and in which I made the changes in collector_sidecar.yml by giving the server URL as (https://graylogserver.com:9000/api/) and after restarting the collector sidecar in first instance I got an error by saying graylog-certificate.pem & graylogkey.pem files were not found in the directory, so then I copied those two files from second instance and kept in the respective path of the first instance and then file not found issue were resolved, but now I am currently getting the below error something related to certificate format,
time="2017-12-11T18:02:09Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put https://graylogserver.com:9000/api/plugins/org.graylog.plugins.collector/collectors/a61675b7-17f3-4b20-8a0c-9d1b929bb53b: x509: certificate signed by unknown authority"
time="2017-12-11T18:02:15Z" level=error msg="[RequestConfiguration] Fetching configuration failed: Get https://graylogserver.com:9000/api/plugins/org.graylog.plugins.collector/a61675b7-17f3-4b20-8a0c-9d1b929bb53b?tags=%5B%22linux%22%2C%22apache%22%2C%22graylogserver%22%5D: x509: certificate signed by unknown authority"
Yes, If I set tls_skip_verify has true and I can able to send data and I could see in the filebeat log file harvest happening,
2017-12-12T08:01:34Z INFO Harvester started for file: /home/ubuntu/graylog.csv
2017-12-12T08:01:34Z WARN SSL/TLS verifications disabled.
2017-12-12T08:02:04Z INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=1 filebeat.harvester.running=1 filebeat.harvester.started=1 libbeat.logstash.call_count.PublishEvents=5 libbeat.logstash.publish.read_bytes=2469 libbeat.logstash.publish.write_bytes=1963971 libbeat.logstash.published_and_acked_events=6747 libbeat.publisher.published_events=6747 publish.events=10002 registrar.states.current=1 registrar.states.update=10002 registrar.writes=5
but the strange thing is I couldn’t find the data in my graylog application.
did you see some counts in your Graylog Interface on the Beats Input that is the target of your collector? Did you check “show messages from that input” and choose “all messages” to check if maybe the time is an issue?
No @jan. I couldn’t see any count in the beats input
did you see some counts in your Graylog Interface on the Beats Input that is the target of your collector? Did you check “show messages from that input” and choose “all messages” to check if maybe the time is an issue?
and also I did checked the show messages in that input but the screen has loading for long time and I couldn’t see any messages.
as you use a self signed certificate, you should give collector sidecar that Information and skip the verification
@jan we are not using self signed certificate and we bought the authorized cert & key from third party.
Is the collector_sidecar configuration needs to be changed related to valid certificates?
Looks like sidecar may not be your problem here at this point but maybe your beat configuration. Make sure that it is correct and shipping as well. If not checking your beat logs may give you insight into where it is failing.
If your beats configuration is making it from graylog to your endpoint server the filebeat configuration should reflect the changes. If not sidecar isn’t communicating with graylog correctly or at all. If the filebeat config is making it from graylog just fine but from there you aren’t seeing logs being shipped then the problem lies within the beat configuration. You have a corresponding beat input setup and listening on the correct port on the graylog side right? Correct input/outputs on the filebeat side? Double check everything. Devil is in the details here.
