Field names prepended with filebeat in new sidecar

Graylog Central (peer support)
1

Hi,

we are in the process of switching to the new sidecars. I configured a sidecar with filebeat (6.1.2 which is the same version as we were running before) and a new beats input. Logs are coming in but all field names are renamed, mostly prepended with filebeat. E.g. filebeat_fields_application instead of application and filebeat_source instead of file. There are also additional fields such as filebeat_@metadata_type

Is this expected behavior of a misconfiguration?

2

I just saw that in the input type there is a box Do not add Beats type as prefix(optional) which solves half my problem. Now it’s just fields_application instead of application. Could this be because of us using an old style filebeat? Here’s my collector configuration:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat:
  prospectors:
  - encoding: plain
    fields:
      application: app-name
      type: log
    ignore_older: 0
    multiline:
      match: after
      negate: true
      pattern: ^20[0-3][0-9]\.
    paths:
    - /var/log/*.log*
    scan_frequency: 10s
    tail_files: false
    type: log
output:
  logstash:
    hosts:
    - ${user.OutputHost}
path:
  data: /var/cache/graylog/collector-sidecar/filebeat/data
  logs: /var/log/graylog/collector-sidecar
tags:
- tag1
- tag2
- tag3
3

Could this be because of us using an old style filebeat?

yes it could.

4

I found the solution. It was not related to the filebeat. The trick is to use fields_under_root also in the prospectors / inputs like that:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat:
  prospectors:
  - encoding: plain
    fields:
      application: app-name
      type: log
    fields_under_root: true
    ignore_older: 0
    multiline:
      match: after
      negate: true
      pattern: ^20[0-3][0-9]\.
    paths:
    - /var/log/*.log*
    scan_frequency: 10s
    tail_files: false
    type: log
output:
  logstash:
    hosts:
    - ${user.OutputHost}
path:
  data: /var/cache/graylog/collector-sidecar/filebeat/data
  logs: /var/log/graylog/collector-sidecar
tags:
- tag1
- tag2
- tag3
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.