Hello,
i just installed the graylog appliance and a rhel7 test client.
i created a filebeats input and installed sidecar on the client.
the client is showing up and appears to be working as it is processing the logs in /var/log
if i click on the sidecar under system/sidecars, i can see all the log files it is reading and
/var/log/messages is one of them… my question is , why dont i see any messages in graylog, from
the messages file from the client…?
there is info being collected from the client, for example /var/log/yum.log.
the messages file line under the sidecar looks like this
i think i found it. it appears to be permissions for the files in ./var/log
for ex… filebeat_log_file_path:/var/log/Xorg.0.log is 644
but messages is 400… it seems that anything 400 i cant search on… but yet in the sidecar view, it shows all the files in /var/log as intake…
so im not sure why i cant search them
scratch that…
that didnt seem to matter… but perhaps its only files that end with .log that seem to be readable…
because /var/log/dmesg is in the list for this sidecar and the permissions are 644 but i cant search it…
but i can search vmware-network.log and that is 400
First - please use the forum tools to make your yml file easier to read.
For instance using the preformatted text tool AND removing all the standard comments that are not needed your .yml boils down to the following… which is much more readable for troubleshooting.
But what I am interested is in the sidecar configuration. The yml file sets the framework for the sidecar but it is the sidecars->configuration that you have applied to the client that defines the specifics of what messages to send into Graylog. Here is a working configuration I have applied to a linux client: (formatted nicely of course:
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
filebeat.inputs:
- type: log
enabled: true
exclude_lines: ['cron\:session']
paths:
- /var/log/auth.log
ignore_older: 72h
tags:
- linux
############# field marker for shunting with stream rules
fields:
beats_os: linux
output.logstash:
hosts:
- ${user.BeatsInput}
path:
data: /var/cache/graylog-sidecar/filebeat/data
logs: /var/log/graylog-sidecar
My first guess is indentation - which is meaningful in YAML files. Make sure the path you want is indented… with (specifically) two spaces, and don’t use tabs…
ok…
the indentations are there…
not sure why it didnt carry over when i pasted it…
im not understanding why all the log files show up under the sidecar but when i search for them, i cant find some of them… for example /var/log/messages and var/log/audit.log are listed but if I do filebeat_source:/var/log/audit.log it returns nothing and if i do filebeat_source:/var/log/boot.log it returns everything in that logfile…
i changed the “paths:” in the filebeat config to say this.
- /var/log/.log
- /var/log/audit/.log
- /var/log/messages
now i am getting audits and messages…
still dont understand why all of /var/log shows up under the sidecar when you click it…
its like an overview of what it claims to be collecting… but yet i cant see all of it…
for example, it claims to have /var/log/tallylog but if i search for it, its not there…
possibly because the config file says /var/log/*.log? and tallylog does not have a .log at the end of it ?
Certainly *.log will only snag files that end in .log… which would mean you would not pick up /var/log/messages. If you want to go into all subdirectories you I think it would be something like /var/log/*/*.log but you would still need a separate line in your sidecar configuration for messages.
