Hello everyone,
I have a pipeline for incoming messages, that adds a field “hour” to each message, so that I can easily filter messages that arrived out of business hours. Unfortunately, this pipeline works differently for messages that arrive through winlogbeat. For these messages the timezone UTC is assumed and the hour field is incorrectly set to UTC time. For all other incoming syslog messages, the correct timezone Europe/Berlin is assumed and hour is set correctly.
The pipeline rule looks like this:
// Tag messages with hour of day
// Check Yoda time for time functions https://www.joda.org/joda-time/
rule "Hour of day"
when
has_field("timestamp")
then
set_field("hour", to_long(to_date($message.timestamp).hourOfDay));
end
Message from winlogbeat with incorrect hour value:
4f4290d1-44fa-11ef-8405-005056ad10dd
Timestamp
2024-07-18 13:38:51.279
Received by
Active Directory Events on
3eed53b1 / XXXXXXX
Stored in index
graylog_366
Routed into streams
All messages
beats_type
winlogbeat
hour
11
message
An account was logged off.
Subject:
Security ID: S-1-5-21-XXXXX-1971
Account Name: XXXXX
Account Domain: XXXXX
Logon ID: 0x19046B6B
Logon Type: 3
This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.
out_of_business_hours
false
source
XXXXXXXX
timestamp
2024-07-18T11:38:51.279Z
weekend
false
All other messages from sysloggers, etc.:
89692a00-44fd-11ef-8405-005056ad10dd
Timestamp
2024-07-18 14:01:59.454
Received by
Syslog UDP on
3eed53b1 / XXXXXXX
Stored in index
graylog_366
Routed into streams
All messages
facility
local4
facility_num
20
hour
14
level
6
message
XXXXXX sandboxd[2098480]: --> [context]zKq7AVICAgAAAP////8Jc2FuZGJveGQAAANZQWxpYnZtYWNvcmUuc28AAI4PNwCieicA0HsnAI4xKACwTCgAS9lJAYJ6AGxpYnB0aHJlYWQuc28uMAAC7y4PbGliYy5zby42AA==[/context]
out_of_business_hours
false
source
XXXXXX
timestamp
2024-07-18T12:01:59.454Z
weekend
false
- OS Information:
Linux XXXXXX 5.15.0-116-generic #126-Ubuntu SMP Mon Jul 1 10:14:24 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
XXX@XXX:~$ timedatectl
Local time: Thu 2024-07-18 14:06:57 CEST
Universal time: Thu 2024-07-18 12:06:57 UTC
RTC time: Thu 2024-07-18 12:06:57
Time zone: Europe/Berlin (CEST, +0200)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
XXX@XXX $ cat /etc/graylog/server/server.conf | grep zone
# The time zone setting of the root user. See http://www.joda.org/joda-time/timezones.html for a list of valid time zons.
#root_timezone = UTC
root_timezone = Europe/Berlin
- Package Version:
XXX@XXX:~$ dpkg -l graylog-server
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii graylog-server 4.2.13-1 all Graylog server
- Service logs, configurations, and environment variables:
3. What steps have you already taken to try and solve the problem?
- Changed timezone on server to UTC
- Added locale to winlogbeat configuration
processors:
- add_locale:
format: abbreviation
- Changed timezone in server.conf to UTC
- Changed rule to include the timezone
set_field("hour", to_long(to_date($message.timestamp, "Europe/Berlin").hourOfDay));
Unfortunately, none of this helps. Can someone point me in the right direction and help me figure out why winlogbeat messages get handled differently by the pipeline rule?
