I have Metricbeat installed and configured through Sidecar and all seems to be working, logs are being pushed into Graylog.
I have two issues to solve:
Firstly the default Metricbeat config seems to have:
path: ${path.config}/modules.d/*.yml
which obviously Graylog errors with as it thinks its a Graylog variable. I’ve replaced this with C:\Program Files\Graylog\sidecar\Metricbeats\modules.d*.yml which is the path to the modules.d directory but is this correct? I’ve not seen any problems with this config but want to make sure its correct.
Secondly even though the logs are coming into Graylog for some reason the source is set to unknown. I have filebeat and winlogbeat installed on the same host and the source is correct, its only incorrect for Metricbeat. I’ve tried adding fields.source: ${sidecar.nodeName} but that doesn’t work. The nodename is set in the sidecar config to use the server hostname (i.e. the config is set to default “”).
My metricbeat config is below. I did see a bug in Github similar to this issue, I’m using metricbeat 7.7.0 and Graylog 3.2.4 in case that’s part of the issue.
In Graylog the Input is setup correctly and is receiving logs from the host on filebeat, winlogbeat and metricbeat. In the logs being shipped the metricbeat service reports the hostname as the correct name, it’s only the source field that’s unknown. I’d like to have this working as default rather than having to configure pipelines or extractors.
I have tried setting fields.source within the config to various different values with no success.
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
metricbeat.config.modules:
# Glob pattern for configuration loading
path: C:\Program Files\Graylog\sidecar\Metricbeat\modules.d\*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
#reload.period: 10s
output.logstash:
# The Logstash hosts
hosts: ["${user.GraylogHost}:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
metricbeat.modules:
- module: system
metricsets:
- cpu # CPU usage
- memory # Memory usage
- network # Network IO
#- process # Per process metrics
#- process_summary # Process summary
- uptime # System Uptime
- socket_summary # Socket summary
#- core # Per CPU core usage
- diskio # Disk IO
#- filesystem # File system usage for each mountpoint
#- fsstat # File system summary metrics
#- raid # Raid
#- socket # Sockets and connection info (linux only)
#- service # systemd service information
enabled: true
period: 10s
Ok, so I’ve downgraded to metricbeats 6.4.2 which is the version of filebeats and winlogbeats that ships with this version of Graylog and source is now reporting correctly.
A secondary issue was that - socket_summary isn’t supported in Windows (which I can’t find documented anywhere) and instead of getting a config file incorrect error I was getting a Service not responding in a timely manner error. When I had -load enabled previously however I had a config file incorrect error. Inconsistent error messages have made this tricky to pin down.
Does anyone know whether the latest version of Graylog (3.3) is compatible with beats 7.7.0?
Just to add to this - another bug between Graylog 3.2 and beats 7.7 is that even though its not been configured the full process logs are sent once a minute from beats to Graylog. So instead of getting the logs I’m expecting (~20) every 10 seconds, I was getting that plus ~300 logs every minute!
