Just migrated from journalbeat to filebeat with journald inputs. I can correctly see the logs on graylog but the source field is not reporting the hostname anymore but “unknown” string, how do I fix that?
Filebeat version: 8.0.0
filebeat.inputs:
- type: journald
id: journald
output.logstash:
hosts: ["myserver:5044"]
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
Hello && Welcome
Looks like you missing some configurations
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
Hope that helps
Thank you @gsmith for joining
I already tried that but it gives me this error
Exiting: error unpacking config data: missing field accessing 'fields.gl2_source_collector'
I noticed with my testing of the version 8 beats that graylog doesn’t set the source right. I’ve got a pipeline rule that sets the source from the Xbeat_host_name field.
I found a workaround
fields_under_root: true
fields:
source: myhost
there’s some filebeat variable I can use without touching the host system variables?
Hello @Azertooth
I’m assuming you downloaded FileBeat and installed it solo? If so, that’s the reason you got the error above, Using those configuration I showed you above, Graylog will place the source for you, but if you downloaded FileBeat solo then yes, you will need to configure you Source in FileBeat configuration file. Or use a pipeline like @kpointer pointed out.
Here is example Graylog Sidecar /w Filebeat
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
filebeat.inputs:
- input_type: log
paths:
- /var/log/*.log
type: log
output.logstash:
hosts: ["10.10.10.10:5044"]
path:
data: /var/lib/graylog-sidecar/collectors/filebeat/data
logs: /var/lib/graylog-sidecar/collectors/filebeat/log
Results:
You are correct, I downloaded filebeat solo!
About the configuration you are suggesting me (I saw that on graylog/sidecar/configuration menu), will it work if my graylog istance is running on another host? I see those path directories are pointing to folders that not exist on my filebeat remote client
Yes,
Graylog sidecar is a wrapper for Winlogbeat, FileBeat, nxlog.
All you have to do is download the sidecar for what OS your using , do a couple configuration to the sidecar configuration file (so you get a connection) then make all your configuration using Graylog Web UI,. Its similar like using Ansible in which you make adjust to files on the Web UI and click send 
I use it on 100 + nodes, if I need to make configurations, it takes 2 seconds to configure all 100+ nodes at once. I mean this does depend on the environment.
wow, it’s brilliant! thank you so much for the insight and help, I will explore this kind of sidecar setup! have a good day <3
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
