Best practice approach with Sidecar

[arcade (Reto Zeder)]

Hey guys,

We are currently migrating to the new 3.0 graylog-sidecar and testing a reasonable way to handle to configuration/collector design. As the previous sidecar used to fetch the configuration based on tags which isn’t the case anymore we are not entirely sure how to create a similar design. The most important thing we need is a custom field with the customer name in it for each log message received by that customer.

Has anyone some design experience which he may share? Do you create one collector e.g winlogbeat and one config per node? Is there a way to send these custom fields via node like it was done before with tags?

We’d appreciate some thoughts on how you handle the sidecar configurations.

Regards
Reto

[tmacgbay]

We have a configuration as follows for Domain Controllers:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["${user.BeatsInput}"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
 - DC
winlogbeat:
  event_logs:
   - name: Application
   - name: System
   - name: Security
   - name: DFS Replication
   - name: Directory Service
   - name: DNS Server
   - name: File Replication Service

And the results are we get a “winlogbeats_tags” field we can work from - here is small a screen capture

[arcade (Reto Zeder)]

Thanks, I had something similar in mind. So you are using just one collector each for windows and linux?

I think in our case we have to create a config file for each application, e.g Domain Controllers and customer. Basically extend your config with a custom field like

fields.customer_name: Customer A

for each customer. This leaves us with quite a lot of configuration files which gets even worse when a customization is needed, e.g for specific system_id.

Is there any chance that the new sidecar is able to handle multiple files at once and merge them together like the old one did like a modular approach? In a worst case scenario you need one individual configuration file for each sidecar.

[tmacgbay]

How about handling the different customers in the pipeline rather than at the OS client. For instance all input from a particular IP range (DNS lookup table on “hostname”…or some other unique thing) is customer X so make a new field CustID: to tag from.

[arcade (Reto Zeder)]

Oh, didn’t consider that approach. That would probably make more sense and is easier to handle, gonna have to check it out. Thanks for that input.