Hello,
1. Describe your incident:
I’m trying to add custom fields with the Windows DHCP Server file log retrieved with filebeat.
The default logs retrieved with Winlogbeat gives only few information but not the leases information nor mac addresses information.
While simulating the extractor or simulating the pipelines rule, it works but the fields are not created on the target stream.
2. Describe your environment:
- Filebeat sidecar configuration:
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
output.logstash:
hosts: ["graylog.company.lan:5044"]
path.data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
path.logs: C:\Program Files\Graylog\sidecar\logs
filebeat.inputs:
- type: log
enabled: true
paths:
- "C:/Windows/System32/dhcp/DhcpSrvLog-*.log"
include_lines: ["^[0-9]{2},"]
fields_under_root: true
Beats input already configured with my previous Winlogbeat configurations.
- RAW Message received for the first time:
11,08/21/23,13:01:09,Renouveler,192.168.1.103,o8-test1.company.lan,005056B2E381,,1245140141,0,,,,,,,,,0
- Grok Pattern Extractor:
^%{INT:dhcpv4.id},(?<timestamp>(%{MONTHNUM}\/%{MONTHDAY}\/%{YEAR},%{HOUR}:%{MINUTE}:%{SECOND})),%{GREEDYDATA:dhcpv4.option.message_type}?((,%{IP:dhcpv4.client_ip},%{DATA:dhcpv4.option.hostname},%{DATA:dhcpv4.client_mac},%{DATA:user.name})|(,,%{DATA:dhcpv4.option.hostname},,)|(,,,,)),%{WORD:dhcpv4.transaction_id},%{WORD:dhcpv4.op_code},,?,,%{DATA:dhcpv4.option.class_identifier},%{DATA:dhcpv4.option.vendor_identifying_options}?((,,,,)|,%{WORD:dhcpv4.option.user_identifying_options},%{WORD:dhcpv4.option.relay_agent_information},,)%{WORD:dhcpv4.option.message}
- Result:
3. What I want to achieve:
The custom fields I want to add are:
- dhcpv4_client_mac_prefix:
I created an extractor to extract the first 6 hexa numbers:
^([\dA-Za-z]{6})
It should create the field: dhcpv4_client_mac_prefix with value 005056 but there is no field created.
- dhcpv4_mac_vendor
I have a list with key-pair value for mac address and vendors.
Lookup Table name: mac_to_vendor
"mac_address";"Manufactor"
"10E992";"INGRAM MICRO SERVICES"
"78F276";"Cyklop Fastjet Technologies (Shanghai) Inc."
"286FB9";"Nokia Shanghai Bell Co., Ltd."
"E0A129";"Extreme Networks Headquarters"
Pipeline
rule "Windows DHCP Server: macaddress to manufactor"
when
has_field("dhcpv4_client_mac_prefix")
then
let update_source = lookup_value("mac_to_vendor", to_string($message.dhcpv4_client_mac_prefix));
set_field("mac_vendor", update_source);
end
I think it does not work because it depends of the previous extractor for dhcpv4_client_mac_prefix
- dhcpv4_op_description:
I have created a lookup table and pipeline for this one:
Lookup Table name: dhcp_op_code
Content:
"dhcpv4_op_code";"dhcpv4_op_description"
"0";"No Quarantine"
"1";"Quarantine"
"2";"Drop Packet"
"3";"Probation"
"6";"No Quarantine Information Probation Time"
Pipeline rule:
rule "Windows Server DHCP: dhcpOPCode and dhcpOPdescription Lookup"
when
has_field("dhcpv4_op_code")
then
let update_source = lookup_value("dhcp_op_code", to_string($message.dhcpv4_op_code));
set_field("dhcpv4_op_description", update_source);
end
- Simulator result:
But it does not appear on the filebeat Stream for new generated logs.
4. How can the community help?
I was wondering if there is an order processing for the extractors ?
Because extracted first is the full DHCP message, then based on the field created, I created a new extractor, and based on the field created, I created pipelines based on it.
Maybe the second extracts look a match on the raw input and not the message ID processed after the first extractor ?
)
