What collector on sidecar use with winlogbeat with a graylog on debian 11?

Hello,

I’m new on graylog’s community, I have installed a Graylog on a debian 11 bullseye.

My problem is, I want to collect all events of my windows server.

First I have installed nxlog on my server sender, but there’s no version of nxlog-ce for debian 11.

Second, I have uninstalled nxlog on windows server, and I want to install winlogbeat, but I don’t what collector use on sidecar? I have only filebeat and nxlog.

Can you help me?

Sorry for my english, I am French but I treat myself :slight_smile:

For Windows machines, you can use WinLogBeat.exe that comes installed with Sidecar. If you choose to stay with beats, you can use filebeat on the debian instance to ship it’s logs into Graylog - this way you would likely only need one Graylog Beats Input to receive it all. Here are some settings for the Graylog side of creating the winlogbeat collector. First the beats log collector:


You can use the default configuration code, to get started, in the collector configuration. Here it is in copyable text:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["${user.BeatsInput}"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
winlogbeat:
  event_logs:
   - name: Application
   - name: System
   - name: Security

hello,
Thanks for your help but that don’t or I do something wrong: The collector winlogbeat exist but I can’t activate it, I’m on debian “C:\Program Files\Graylog\sidecar\winlogbeat.exe” don’t exist.
What config I do on debian?
On Collector administration, I have only filebeat and nxlog

If I understood correctly, I made a filebeat conf on filebeat collector by default on graylog server, and on my windows server who send log, I installed winlogbeat, that is correct?

My actual config is on graylog:

  • input on port 5044
  • collector filebeat with conf file linux by default (only change ip address)

On client: in winlogbeat.yml
only change
#output.logstash:

The Logstash hosts

hosts: [“192.168.xxx.xxx:5044”]

winlogbeat (and C:\Program Files\Graylog\sidecar\winlogbeat.exe) is exclusive to the Windows environment. for Debian, you would use filebeat. You mention that as well in your last post… I am not sure where the confusion is. They both require a sidecar.yml that is set up correctly to point to your Graylog server. On windows or linux, you don’t need to create a beat service, you create a sidecar service that handles starting stopping and configuring your beats application (winlogbeat or filebeat) from the Graylog GUI. Maybe read through this again - there are things it misses but it does have a good base explanation. Here is an example sidecar.yml to work with (Note that YML is sensitive to spacing):

server_url: http://<gralog-server-name>:9000/api/
server_api_token: "<api_token>" 
update_interval: 10
tls_skip_verify: true
send_status: true
list_log_files:
collector_id: file:C:\Program Files\Graylog\sidecar\collector-id
cache_path: C:\Program Files\Graylog\sidecar\cache
log_path: C:\Program Files\Graylog\sidecar\logs
log_rotation_time: 86400
log_max_age: 604800
tags: [windows]
collector_binaries_accesslist: []
backends:
    - name: nxlog
      enabled: false
      binary_path: C:\Program Files (x86)\nxlog\nxlog.exe
      configuration_path: C:\Program Files\Graylog\sidecar\generated\nxlog.conf
    - name: winlogbeat
      enabled: true
      binary_path: C:\Program Files\Graylog\sidecar\winlogbeat.exe
      configuration_path: C:\Program Files\Graylog\sidecar\generated\winlogbeat.yml
    - name: filebeat
      enabled: true
      binary_path: C:\Program Files\Graylog\sidecar\filebeat.exe
      configuration_path: C:\Program Files\Graylog\sidecar\generated\filebeat.yml
    - name: auditbeat
      enabled: false
      binary_path: C:\Program Files\Graylog\sidecar\auditbeat.exe
      configuration_path: C:\Program Files\Graylog\sidecar\generated\auditbeat.yml

Hello,
The confusion is on the sidecar.yml example, all path goes to C:\program files…

What is the better way for collect all windows event on my graylog server? I have installed for test nxlog on a server windows and it send log without sidecar, only an input.
What is the gain in this way to compare Sidecar associated with FileBeat and WinLogbeat?

Here is my sidecar.yml:

#The URL to the Graylog server API.
server_url: "http://xxx.xxx.x.xxx:9000/api/" (ip address of graylog server)

#The API token to use to authenticate against the Graylog server API.
#This field is mandatory
server_api_token: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

#The node ID of the sidecar. This can be a path to a file or an ID string.
#If set to a file and the file doesn't exist, the sidecar will generate an
#unique ID and writes it to the configured path.
#
#Example file path: "file:/etc/graylog/sidecar/node-id"
#Example ID string: "6033137e-d56b-47fc-9762-cd699c11a5a9"
#
#ATTENTION: Every sidecar instance needs a unique ID!
#
#node_id: "file:/etc/graylog/sidecar/node-id"

#The node name of the sidecar. If this is empty, the sidecar will use the
#hostname of the host it is running on.
#node_name: ""

#The update interval in seconds. This configures how often the sidecar will
#contact the Graylog server for keep-alive and configuration update requests.
#update_interval: 10

#This configures if the sidecar should skip the verification of TLS connections.
#Default: false
#tls_skip_verify: false

#This enables/disables the transmission of detailed sidecar information like
#collector statues, metrics and log file lists. It can be disabled to reduce
#load on the Graylog server if needed. (disables some features in the server UI)
#send_status: true

#A list of directories to scan for log files. The sidecar will scan each
#directory for log files and submits them to the server on each update.
#
#Example:
#    list_log_files:
#      - "/var/log/nginx"
#      - "/opt/app/logs"
#
#Default: empty list
#list_log_files: []

#Directory where the sidecar stores internal data.
#cache_path: "/var/cache/graylog-sidecar"

#Directory where the sidecar stores logs for collectors and the sidecar itself.
#log_path: "/var/log/graylog-sidecar"

#The maximum size of the log file before it gets rotated.
#log_rotate_max_file_size: "10MiB"

#The maximum number of old log files to retain.
#log_rotate_keep_files: 10

#Directory where the sidecar generates configurations for collectors.
#collector_configuration_directory: "/var/lib/graylog-sidecar/generated"

#A list of binaries which are allowed to be executed by the Sidecar. An empty list disables the whitelist feature.
#Wildcards can be used, for a full pattern description see https://golang.org/pkg/path/filepath/#Match
#Example:
#    collector_binaries_whitelist:
#      - "/usr/bin/filebeat"
#      - "/opt/collectors/*"
#
#Example disable whitelisting:
#collector_binaries_whitelist: []
#
#Default:
#collector_binaries_whitelist:
#- "/usr/bin/filebeat"
#- "/usr/bin/packetbeat"
#- "/usr/bin/metricbeat"
#- "/usr/bin/heartbeat"
#- "/usr/bin/auditbeat"
#- "/usr/bin/journalbeat"
#- "/usr/share/filebeat/bin/filebeat"
#- "/usr/share/packetbeat/bin/packetbeat"
#- "/usr/share/metricbeat/bin/metricbeat"
#- "/usr/share/heartbeat/bin/heartbeat"
#- "/usr/share/auditbeat/bin/auditbeat"
#- "/usr/share/journalbeat/bin/journalbeat"
#- "/usr/bin/nxlog"
#- "/opt/nxlog/bin/nxlog"

Herre is filebeat.conf:
#Needed for Graylog
fields_under_root: true
fields.collector_node_id: graylog
fields.gl2_source_collector: eb36f963-1bcc-4413-8359-a19022a9ca9a

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/*.log
  type: log
output.logstash:
   hosts: ["xxx.xxx.xxx.xxx:5044"]
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

And on windows side, my winlogbeat.yml
######################Winlogbeat Configuration Example ########################

#This file is an example configuration file highlighting only the most common
#options. The winlogbeat.reference.yml file from the same directory contains
#all the supported options with more comments. You can use it as a reference.
#
#You can find the full configuration reference here:
#https://www.elastic.co/guide/en/beats/winlogbeat/index.html

#======================== Winlogbeat specific options =========================

#event_logs specifies a list of event logs to monitor as well as any
#accompanying options. The YAML data type of event_logs is a list of
#dictionaries.
#
#The supported keys are name, id, xml_query, tags, fields, fields_under_root,
#forwarded, ignore_older, level, event_id, provider, and include_xml.
#The xml_query key requires an id and must not be used with the name,
#ignore_older, level, event_id, or provider keys. Please visit the
#documentation for the complete details of each option.
#https://go.es.io/WinlogbeatConfig

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security

  - name: Microsoft-Windows-Sysmon/Operational

  - name: Windows PowerShell
    event_id: 400, 403, 600, 800

  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106

  - name: ForwardedEvents
    tags: [forwarded]

#====================== Elasticsearch template settings =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false


#================================== General ===================================

#The name of the shipper that publishes the network data. It can be used to group
#all the transactions sent by a single shipper in the web interface.
#name:

#The tags of the shipper are included in their own field with each
#transaction published.
#tags: ["service-X", "web-tier"]

#Optional fields that you can specify to add additional information to the
#output.
#fields:
#env: staging

#================================= Dashboards =================================
#These settings control loading the sample dashboards to the Kibana index. Loading
#the dashboards is disabled by default and can be enabled either by setting the
#options here or by using the `setup` command.
#setup.dashboards.enabled: false

#The URL from where to download the dashboards archive. By default this URL
#has a value which is computed based on the Beat name and version. For released
#versions, this URL points to the dashboard archive on the artifacts.elastic.co
#website.
#setup.dashboards.url:

#=================================== Kibana ===================================

#Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
#This requires a Kibana endpoint configuration.
setup.kibana:

  #Kibana Host
  #Scheme and port can be left out and will be set to the default (http and 5601)
  #In case you specify and additional path, the scheme is required: http://localhost:5601/path
  #IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  #host: "localhost:5601"

  #Kibana Space ID
  #ID of the Kibana Space into which the dashboards should be loaded. By default,
  #the Default Space will be used.
  #space.id:

#=============================== Elastic Cloud ================================

#These settings simplify using Winlogbeat with the Elastic Cloud (https://cloud.elastic.co/).

#The cloud.id setting overwrites the `output.elasticsearch.hosts` and
#`setup.kibana.host` options.
#You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:

#The cloud.auth setting overwrites the `output.elasticsearch.username` and
#`output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:

#================================== Outputs ===================================

#Configure what output to use when sending the data collected by the beat.

#---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  #Array of hosts to connect to.
  hosts: ["localhost:9200"]

  #Protocol - either `http` (default) or `https`.
  #protocol: "https"

  #Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  #username: "elastic"
  #password: "changeme"

  #Pipeline to route events to security, sysmon, or powershell pipelines.
  pipeline: "winlogbeat-%{[agent.version]}-routing"

#------------------------------ Logstash Output -------------------------------
#output.logstash:
  #The Logstash hosts
  hosts: ["xxx.xxx.xxx.xxx:5044"] 

  #Optional SSL. By default is off.
  #List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  #Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  #Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

#================================== Logging ===================================

#Sets log level. The default log level is info.
#Available log levels are: error, warning, info, debug
#logging.level: debug

#At debug level, you can selectively enable logging only for some components.
#To enable all selectors use ["*"]. Examples of other selectors are "beat",
#"publisher", "service".
#logging.selectors: ["*"]

#============================= X-Pack Monitoring ==============================
#Winlogbeat can export internal metrics to a central Elasticsearch monitoring
#cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The
#reporting is disabled by default.

#Set to true to enable the monitoring reporter.
#monitoring.enabled: false

#Sets the UUID of the Elasticsearch cluster under which monitoring data for this
#Winlogbeat instance will appear in the Stack Monitoring UI. If output.elasticsearch
#is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch.
#monitoring.cluster_uuid:

#Uncomment to send the metrics to Elasticsearch. Most settings from the
#Elasticsearch output are accepted here as well.
#Note that the settings should point to your Elasticsearch *monitoring* cluster.
#Any setting that is not set is automatically inherited from the Elasticsearch
#output configuration, so if you have the Elasticsearch output configured such
#that it is pointing to your Elasticsearch monitoring cluster, you can simply
#uncomment the following line.
#monitoring.elasticsearch:

#============================== Instrumentation ===============================

#Instrumentation support for the winlogbeat.
#instrumentation:
    #Set to true to enable instrumentation of winlogbeat.
    #enabled: false

    #Environment in which winlogbeat is running on (eg: staging, production, etc.)
    #environment: ""

    #APM Server hosts to report instrumentation results to.
    #hosts:
    # - http://localhost:8200

    #API Key for the APM Server(s).
    #If api_key is set then secret_token will be ignored.
    #api_key:

    #Secret token for the APM Server(s).
    #secret_token:


#================================= Migration ==================================

#This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true

I don’t know where is the problem, thanks for helping a noob of graylog :wink:

Hello @chnick

When posting configuration files or code, please use the Markdown. I have adjusted you post to reflect this. :+1:
You can find more here

The advantage of sidecar is you can push a single consistent configuration to multiple client sidecar machines using the Graylog GUI. Using nxlog or beats is a preference, both of them can be manged (start, stop, configured…) from Graylog once you have it connected. Review the documentation, it explains well…

OK thanks for all.

I finally reinstall ubuntu 20.04 in place of debian 11 for the compatibility with nxlog.

I am not sure that is required, Ubuntu is an offshoot of Debian… Where did you see that nxlog is incompatible with Debian?

Hello, but nxlog ce is available in ubuntu 20.04 and not on debian 11. (cf: NXLog Community Edition - Downloads | nxlog.co)

Thanks for your help and for the lost time, but that’s help me in my new installation.

I have installed an ubuntu 20.04 and I don’t have meesage, I open a new topic :wink:

Hello @chnick

Nxlog or Winlogbeat is your preferred shippers for Windows operating systems.

If Graylog is on Debian 11 you can use Rsyslog which I believe is native for Linux OS’s or install FileBeat which I believe is a preferred log shipper for this OS.

There is no need to open a new topic , if your still having problems with messages NOT being received this would be the best place to finish troubleshooting your issue.

Edit: What we need is the following to help us ,Help you.

  1. What is your configure input and the settings for this?
  2. How did you configure log shipper?
  3. Are you using a loopback IP address ( 127.0.0.1) or public IP address in you Graylog configuration file?
  4. Is there a firewall enabled?
  5. By chance either Selinux or AppArmor enabled?
  6. What is the status of Elasticsearch/Graylog? (curious if there are any errors shown).
  7. Anything pertaining to this issue within you Logs files?
1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.