DHCP Logging to graylog using nxlog

[hagge10 (hagge)]

Hi!

I am trying to get the DHCP logs over to graylog from a Windows 2016 server, if tried to follow every suggestion related to DHCP in regards to nxlog and graylog but im just not able to get it to work,

My nxlog.conf setup is:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules 
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
Loglevel INFO

<Extension gelf> 
    Module xm_gelf
</Extension>

<Extension ParseDHCP>
    Module  xm_csv
        
        Fields $ID ,$Date ,$Time ,$Description ,$IPAddress ,$ReportedHostname ,$MACAddress ,$UserName ,$TransactionID ,$QResult ,$Probationtime ,$CorrelationID ,$Dhcid ,$VendorClass(Hex) ,$VendorClass(ASCII) ,$UserClass(Hex) ,$UserClass(ASCII) ,$RelayAgentInformation ,$DnsRegError
    
    Delimiter   ','
</Extension>

<Extension json> 
    Module xm_json
</Extension>

<Input DHCP_IN> 
    Module  im_file
    File    "C:\\Windows\\Sysnative\\dhcp\\DhcpSrvLog-*.log"
    SavePos TRUE
    InputType   LineBased
    Exec    $Message = $raw_event;
    
    Exec if $raw_event =~ /^[0-9][0-9],/                        \
            {                                                       \
                ParseDHCP->parse_csv();                             \
                if $raw_event =~ /^00/ $IDdef = "The log was started."; \
                if $raw_event =~ /^01/ $IDdef = "The log was stopped."; \
                if $raw_event =~ /^02/ $IDdef = "The log was temporarily paused due to low disk space.";    \
                if $raw_event =~ /^10/ $IDdef = "A new IP address was leased to a client."; \
                if $raw_event =~ /^11/ $IDdef = "A lease was renewed by a client."; \
                if $raw_event =~ /^12/ $IDdef = "A lease was released by a client.";    \
                if $raw_event =~ /^13/ $IDdef = "An IP address was found to be in use on the network."; \
                if $raw_event =~ /^14/ $IDdef = "A lease request could not be satisfied because the scope's address pool was exhausted.";   \
                if $raw_event =~ /^15/ $IDdef = "A lease was denied.";  \
                if $raw_event =~ /^16/ $IDdef = "A lease was deleted."; \
                if $raw_event =~ /^17/ $IDdef = "A lease was expired and DNS records for an expired leases have not been deleted."; \
                if $raw_event =~ /^18/ $IDdef = "A lease was expired and DNS records were deleted.";    \
                if $raw_event =~ /^20/ $IDdef = "A BOOTP address was leased to a client.";  \
                if $raw_event =~ /^21/ $IDdef = "A dynamic BOOTP address was leased to a client.";  \
                if $raw_event =~ /^22/ $IDdef = "A BOOTP request could not be satisfied because the scope's address pool for BOOTP was exhausted."; \
                if $raw_event =~ /^23/ $IDdef = "A BOOTP IP address was deleted after checking to see it was not in use.";  \
                if $raw_event =~ /^24/ $IDdef = "IP address cleanup operation has began.";  \
                if $raw_event =~ /^25/ $IDdef = "IP address cleanup statistics.";   \
                if $raw_event =~ /^30/ $IDdef = "DNS update request to the named DNS server.";  \
                if $raw_event =~ /^31/ $IDdef = "DNS update failed.";   \
                if $raw_event =~ /^32/ $IDdef = "DNS update successful.";   \
                if $raw_event =~ /^33/ $IDdef = "Packet dropped due to NAP policy.";    \
                if $raw_event =~ /^34/ $IDdef = "DNS update request failed.as the DNS update request queue limit exceeded.";    \
                if $raw_event =~ /^35/ $IDdef = "DNS update request failed.";   \
                if $raw_event =~ /^36/ $IDdef = "Packet dropped because the server is in failover standby role or the hash of the client ID does not match.";   \
                if $raw_event =~ /^[5-9][0-9]/ $IDdef = "Codes above 50 are used for Rogue Server Detection information.";  \
                if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,0,/ $QResultDef = "NoQuarantine";    \
                if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,1,/ $QResultDef = "Quarantine";  \
                if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,2,/ $QResultDef = "Drop Packet"; \
                if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,3,/ $QResultDef = "Probation";   \
                if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,6,/ $QResultDef = "No Quarantine Information ProbationTime:Year-Month-Day Hour:Minute:Second:MilliSecond.";  \
                $host           =   hostname_fqdn();                \
                $EventTime      =   parsedate($Date + " " + $Time); \
                $SourceName     =   "DHCPEvents";                   \
            $Message        =   to_json();                      \
                }                                                       \
                else                                                    \
            drop();

</Input>

<Output DHCP_Out>
    Module      om_udp
        OutputType  GELF
        Host        greylogserverip
        Port        5441
</Output>

<Route DHCP>
    Path        DHCP_IN => DHCP_OUT
</Route>

I have tried with both the content pack “WinDHCPLogs” and setting up a basic input on GELF UDP on the graylog server, no errors but looking at the input nothing is shipped from the server to Graylog.

Any suggestions? Thanks!

[jochen]

What type of input have you created in Graylog?
Please post the complete configuration of that input.

[hagge10 (hagge)]

Hi Jochen, thank you for replying

My input is:

GELF UDP:

bind_address: 0.0.0.0
decompress_size_limit: 8388608
override_source:
port: 5441
recv_buffer_size: 1048576

[jtkarvo]

First: remove the to_json() line. When sending in GELF, it is not needed.

Then either check that the fields in the log files are exactly the same as in the parse_csv function, or alternatively remove the parse_csv stuff and select another way of parsing the log lines.

If these steps do not help, remove parts of the nxlog configuration until you get something to Graylog - then start adding stuff back. Also see the nxlog.log file for any configuration errors.

[hagge10 (hagge)]

Hi jtkarvo, thanks for replying.

ive removed the majority of the code now but to no success, whats left is:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules 
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
Loglevel debug

<Extension gelf> 
    Module xm_gelf
</Extension>

<Input DHCP_IN> 
    Module  im_file
    File    "C:\\Windows\\Sysnative\\dhcp\\DhcpSrvLog-*.log"
    SavePos TRUE
    InputType   LineBased
</Input>

<Output DHCP_Out>
    Module      om_udp
        OutputType  GELF
        Host        graylogserverIP
        Port        5441
</Output>

<Route DHCP>
    Path        DHCP_IN => DHCP_OUT
</Route>

when settings loglevel to debug the only think that even resembles error is this:

2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Fri.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Fri.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Mon.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Mon.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Sat.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Sat.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Sun.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Sun.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Thu.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Thu.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Tue.log
2018-02-20 19:46:18 DEBUG got EOF for C:\Windows\Sysnative\dhcp\DhcpSrvLog-Tue.log
2018-02-20 19:46:18 DEBUG Module DHCP_IN got EOF from C:\Windows\Sysnative\dhcp\DhcpSrvLog-Wed.log

Thanks again for trying to help out!

[jtkarvo]

Clearly nxlog finds the files. You config seems simple enough that you can debug now; after getting raw lines in, you can start getting stuff back.

Check that the input files are not empty.

Also, check that Graylog input, if the bytes received grows. If it grows, but the lines do not appear as messages, you can try to see from the Graylog side if you see any errors.

You can also try using tcpdump/wireshark also on the sending end to see if nxlog sends anything. You can also try switching to GELF_TCP and om_tcp, and a tcp gelf input in Graylog to see if nxlog actually can connect to Graylog (you can see how many connections are establishded on the Graylog inputs page).