Hello Everyone,
This must be the biggest noob question. I have my sidecar/filebeat running and it appears to be collecting logs and current logs. I am not sure how to see this data. I go to search and follow the guide on how query syntax and I do not see anything. Am I missing something?
he @CloudBeard
I think that you only see the list of available files on the remote server where sidecar is installed. This is not what is collected and transported to Graylog. This is just a directory listing.
That makes sense, I went back through the documentation to ensure I followed the steps correctly or what I think is correct.
I created a filebeat on linux collector with the following config in the webui
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
filebeat.inputs:
- input_type: log
paths:
- /var/log/*.log
type: log
output.logstash:
hosts: ["192.168.2.61:5044"]
ssl.certificate_authorities: ["/etc/ssl/certs/ca.pem"]
ssl.certificate: "/etc/ssl/certs/machine-cert.pem"
ssl.key: "/etc/ssl/certs/machine-key.pem"
path:
data: /var/lib/graylog-sidecar/collectors/filebeat/data
logs: /var/lib/graylog-sidecar/collectors/filebeat/log
and I have it attached to a running input beat. I can see that the collector/sidecar/input are all running. This is evident in that it shows the dirs but I am not sure what or how to get the logs. - probably skipped a step.
Found somewhat helpful data at /var/lib/graylog-sidecar/collectors/filebeat/log/filebeat looks like it is failing to connect to my host at 192.168.2.61:5044
2020-05-27T10:07:32.328-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://log.domain.com:5044)) with 3 reconnect attempt(s)
This is what the logs say, I have swapped the host section in the above config file from IP to DNS name, same error for both ways just shows dns name or IP.
you should start without securing tcp. there are many people fighting with security.
you can also try to run filebeat with -e -v
telnet host/ip 5044 ?
Would that just be commenting out the security bits in the server.conf and related files?
Just disable SSL/TLS between filebeat and gl-input
output.logstash section in filebeat configuration and graylog input, if you have some.
Just noticed there are differences between my filebeat.yml and the configuration on the collector in the webui.
seem a / was missing in one of the cert fields
enabled: false is in the yml file but not the webui
The following from the webui are missing:
fields_under_root: true
fields.collector_node_id: {sidecar.nodeName}
fields.gl2_source_collector: {sidecar.nodeId}
I think maybe directory permissions could be my issue? Not able to write to the filebeat yml
Do you have correct filebeat configuration assigned in the manage sidecar section?
Yes, just double checked to make sure. I thought the same after I noticed the differences lol.
and what about sidecar.log?
2020-05-27T11:03:58.546-0400 ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://log.domain.com:5044)): read tcp 192.168.2.61:58074->192.168.2.61:5044: read: connection reset by peer
2020-05-27T11:03:58.547-0400 INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://log.domain.com:5044)) with 36 reconnect attempt(s)
Also just noticed
I will try to unmount it the config from the sidecar and make some modifications and see if it works.
I created a new collector configuration and moved the sidecar over to it. Shows success and it is running but when I check out the /etc/filebeatfilebeat.yml it still has the old config. Logs still say the. Do you know what the permissions should be by chance?
Unfortunately i cant help you with this. Im using sidecars only in windows enviroment.
Thanks for your help up to this point, not sure I would have known where to look so far! I will start checking online and see if I can find more from this point and report back if I make any progress.
or you can try to install filebeat from repo and configure it without using the sidecar (just for now/test)
filebeat.yml configuration is the same without the GL head
Back to square one. I have not been able to figure out anything permission related. I also tried editing the filebeat individually and trying that, no luck.
I don’t think it is permission related. I followed what Fangycz said earlier and take out the security settings between filebeat and server and I can see the messages now. Has to be something I have done setting up the security between the server and filebeats.
Closing issue will create another post related to the security settings after I give it another go and I get stuck.
