Pipeline - Create field with lookup tables


I am using nxlog and sidecar to ingest Windows DHCP Server Logs into Graylogs.

The logs are automatically parsed thanks to GELF format but there is no EventCategory sent by Windows.

I’ve followed this tutorial: How to use graylog lookup table

I have created the lookup table to match EventID with EventCategory.

  • event.csv

And it is working:
EventID 107 lookup with EventCategory DHCPv4.ReservationDeleted

Now I’m looking to create a pipeline to add a new field “EventCategory” where the value will be written according to EventID value.

I tried this Pipeline rule but the rule simulation gives no result:

rule "Windows Server DHCP: EventID lookup with EventCategory"

let update_source = lookup_value("eventID_eventCategory",$message.EventID);
set_field("EventCategory", update_source);


Am I missing something ?
Thank you !

OS & Package Information:

  • Rocky Linux 9
  • docker compose 2.18.1
  • Graylog 5.1.2
  • traefik 2.10.1
  • mongodb 6.0.6
  • elasticsearch 7.10.2
  • Windows Server DHCP
  • nxlog latest
  • sidecar latest

Hey @s0p4L1N

The piepline look good. How does you Data Adaptor look? By default "," the delimiter to use for separating entries.Looks like your using ";" instead.

Here’s my data adapter config:

  • I use ; as a separator in my case.

  • When I try the pipeline simulator in Pipelines > Simulator

  • I select the DHCP Stream

  • RAW Message:

  "Task": 0,
  "Keywords": -9223372036854776000,
  "EventType": "INFO",
  "collector_node_id": "srvad1",
  "gl2_remote_ip": "",
  "gl2_remote_port": 50586,
  "Opcode": "Informations",
  "IP_Name": "[[]company.lan]",
  "source": "SRVAD1.COMPANY.lan",
  "gl2_source_input": "64a3c7a431459c203793e12c",
  "ClientName": "COMPANY\\username",
  "SeverityValue": 2,
  "Version": 0,
  "UserID": "S-1-5-21-3256819886-XXXXXXXXX-XXXXXXXXXX-XXX",
  "gl2_source_node": "3b147713-efd6-45b0-83e4-f3b8aeea69ef",
  "ProcessID": 2404,
  "timestamp": "2023-07-04T15:35:32.000Z",
  "gl2_accounted_message_size": 977,
  "OpcodeValue": 0,
  "gl2_source_collector": "78fd7c59-b789-4596-be22-1754f8a179cf",
  "SourceModuleType": "im_msvistalog",
  "level": 6,
  "Channel": "Microsoft-Windows-Dhcp-Server/Operational",
  "streams": [
  "gl2_message_id": "01H4GRPTD0000016SAPK1ZCAVB",
  "SourceName": "Microsoft-Windows-DHCP-Server",
  "Severity": "INFO",
  "message": "La réservation [[]] pour IPv4 est supprimée sous l’étendue [[]company.lan] par COMPANY\\username.",
  "AccountType": "User",
  "EventReceivedTime": "2023-07-04 17:35:33",
  "SourceModuleName": "dhcp_server_eventlog",
  "ProviderGuid": "{6D64F02C-A125-4DAC-9A01-F0555B41CA84}",
  "full_message": "La réservation [[]] pour IPv4 est supprimée sous l’étendue [[]company.lan] par COMPANY\\username.",
  "ThreadID": 11076,
  "EventID": 107,
  "ReservationName": "[[]]",
  "_id": "80143c92-1a80-11ee-858f-0242ac150005",
  "Domain": "COMPANY",
  "RecordNumber": 134,
  "AccountName": "username"
  • Message input:


  • Message codec:


But now that I understand by writing my message, I just forgot to add the pipeline and the stage in order to the simulator to work !

I though the simulator was here to test the rule before applying it to any stream ?
But instead you need to create the pipeline and add the rule to then test it with the simulator.

Anyway, it works :slight_smile:

Hey @s0p4L1N

Awesome :+1: Some time it helps just talking about it.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.