Filebeat unknown source field

Just migrated from journalbeat to filebeat with journald inputs. I can correctly see the logs on graylog but the source field is not reporting the hostname anymore but “unknown” string, how do I fix that?

Filebeat version: 8.0.0

  - type: journald
    id: journald
  hosts: ["myserver:5044"]
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Hello && Welcome

Looks like you missing some configurations

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

Hope that helps

Thank you @gsmith for joining

I already tried that but it gives me this error

Exiting: error unpacking config data: missing field accessing 'fields.gl2_source_collector'

I noticed with my testing of the version 8 beats that graylog doesn’t set the source right. I’ve got a pipeline rule that sets the source from the Xbeat_host_name field.

I found a workaround

Needed for Graylog

fields_under_root: true
  source: myhost

there’s some filebeat variable I can use without touching the host system variables?

1 Like

Hello @Azertooth

I’m assuming you downloaded FileBeat and installed it solo? If so, that’s the reason you got the error above, Using those configuration I showed you above, Graylog will place the source for you, but if you downloaded FileBeat solo then yes, you will need to configure you Source in FileBeat configuration file. Or use a pipeline like @kpointer pointed out.

Here is example Graylog Sidecar /w Filebeat

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

- input_type: log
    - /var/log/*.log
  type: log
   hosts: [""]
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log


Hope that helps

You are correct, I downloaded filebeat solo!
About the configuration you are suggesting me (I saw that on graylog/sidecar/configuration menu), will it work if my graylog istance is running on another host? I see those path directories are pointing to folders that not exist on my filebeat remote client

Graylog sidecar is a wrapper for Winlogbeat, FileBeat, nxlog.
All you have to do is download the sidecar for what OS your using , do a couple configuration to the sidecar configuration file (so you get a connection) then make all your configuration using Graylog Web UI,. Its similar like using Ansible in which you make adjust to files on the Web UI and click send :smiley:
I use it on 100 + nodes, if I need to make configurations, it takes 2 seconds to configure all 100+ nodes at once. I mean this does depend on the environment.

1 Like

wow, it’s brilliant! thank you so much for the insight and help, I will explore this kind of sidecar setup! have a good day <3

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.