Unknown source with filebeat

[romgo]

Hi,

I just deployed a graylog server in version 3.0.2.
I created a beats input.

I added a client with filebeat version 7.5.1, in graylog webui I can see the logs showing up, but they are all showing source as unknown.

When I look at the differents fields I can see that the hostname of client is present in : filebeat_agent_name
filebeat_agent_hostname,
filebeat_host_name.

is there a way to configure graylog or filebeat in order to set the source filed to hostname ?

Best regards,

[tmacgbay]

Here is an example filebeats collector configuration - your issue may be needing the first three/four lines in it… maybe post your collector configuration (properly formatted of course).

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: 
   - ${user.BeatsInput}
   ssl:
   verification_mode: none
path:
  data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
filebeat:
  inputs:
    - type: log
      paths:
        - C:\Windows\Logs\DNSServer\dns.log

[romgo]

Hi,

thank you for your feed back.
I wasn’t able to use this config on my filebeat. But we configured some extractors :

{
  "extractors": [
    {
      "title": "LogLevel",
      "extractor_type": "regex",
      "converters": [],
      "order": 1,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "log_level",
      "extractor_config": {
        "regex_value": "\\d\\d\\d\\d-\\d\\d-\\d\\d\\s\\d\\d:\\d\\d:\\d\\d\\s(?<loglevel>\\w+)\\s.*"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "Timestamp",
      "extractor_type": "regex",
      "converters": [
        {
          "type": "date",
          "config": {
            "date_format": "yyyy-MM-dd HH:mm:ss",
            "time_zone": "Europe/Paris",
            "locale": "fr-FR"
          }
        }
      ],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "timestamp",
      "extractor_config": {
        "regex_value": "(?<date>\\d{4}-\\d{2}-\\d{2}\\s\\d\\d\\:\\d\\d\\:\\d\\d)"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "Source",
      "extractor_type": "copy_input",
      "converters": [],
      "order": 2,
      "cursor_strategy": "copy",
      "source_field": "filebeat_host_name",
      "target_field": "source",
      "extractor_config": {},
      "condition_type": "none",
      "condition_value": ""
    }
  ],
  "version": "3.0.2"
}

on the input.