Sidecar-Configuration on Ubunut-Linux

Graylog Central (peer support)
1

1. Describe your incident:

I would like to collect Webserver-Logs from a Ubuntu 20.04 LTS Server running nginx via Sidecar/Filebeat. Installation and Configuration of Sidecare succeeded - the Server is listed in the Sidecar-Window on the Graylog-Webinterface. The Process is running. But no Logs are collected.

2. Describe your environment:

  • OS Information:
    Graylog 5.2.3 in Docker-Container; Ubuntu 20.04 LTS Docker-Host

  • Service logs, configurations, and environment variables:
    Sidecar-Configuration in Grayog:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}


output.logstash:
   hosts: ["${user.graylog_host}:5044"]
path:
   data: ${sidecar.spoolDir!"/var/lib/graylog-sidecar/collectors/filebeat"}/data
   logs: ${sidecar.spoolDir!"/var/lib/graylog-sidecar/collectors/filebeat"}/log

filebeat.inputs:

- type: filestream
  enabled: true
  paths:
  - /var/log/nginx/access_intranet.log
  - /var/log/nginx/error_intranet.log
 
  fields_under_root: true

Latest Log-Lines from /var/logs/graylog/sidecar.log from the local machine where i would like to collect the Logs from:

time="2024-01-19T09:24:18+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Configuration change detected, rewriting configuration file."
time="2024-01-19T09:24:19+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Stopping"
time="2024-01-19T09:24:19+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Starting (exec driver)"
time="2024-01-19T09:28:30+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Got remote stop command"
time="2024-01-19T09:28:30+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Stopping"
time="2024-01-19T09:29:10+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Got remote start command"
time="2024-01-19T09:29:10+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Starting (exec driver)"
time="2024-01-19T14:27:32+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Configuration change detected, rewriting configuration file."
time="2024-01-19T14:27:33+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Stopping"
time="2024-01-19T14:27:33+01:00" level=info msg="[filebeat-65aa2d9ab99f932dda3fdbdb] Starting (exec driver)"

3. What steps have you already taken to try and solve the problem?

I’ve used this to install Graylog Sidecar: Graylog Sidecar

In addition i’ve manually downloaded the filebeat.deb-Package an used dpkg -i to install it on the Ubuntu-System. I’ve added the Filebeat-Configuration on Graylog-WebInterface and assigned it to the System.

4. How can the community help?

I do not know where to look to troubleshoot the issue. May anyone can help troubelshooting?

2

hey @hoehr-grenzhausen

Try tcpdump on Graylog node this might give an an idea what sgoing on. If logs are reaching Graylog from the ubuntu instance then it might be a configuration or something. If not, then it would be on the ubuntu side. Ensure Graylog sidecar can access nginx logs. It might take a couple minutes to get started and dont for get Date/Time correct this sometime is common cause of not seeing logs right away.
Hope that helps.