Graylog shows eventlog from a machine but not IIS logs


(Zloy) #1

Hi there!
I’m trying to get IIS logs to graylog. Here is my nxlog.conf

Panic Soft
#NoFreeOnExit TRUE

define ROOT     C:\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf
define LOGDIR   %ROOT%\data
define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _charconv>
    Module      xm_charconv
    AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>

<Extension _exec>
    Module      xm_exec
</Extension>

<Extension gelf>
    Module      xm_gelf
</Extension>

<Extension _fileop>
    Module      xm_fileop

    # Check the size of our log file hourly, rotate if larger than 5MB
    <Schedule>
        Every   1 hour
        Exec    if (file_exists('%LOGFILE%') and \
                   (file_size('%LOGFILE%') >= 5M)) \
                    file_cycle('%LOGFILE%', 8);
    </Schedule>

    # Rotate our log file every week on Sunday at midnight
    <Schedule>
        When    @weekly
        Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
    </Schedule>
</Extension>

<Input internal>
    Module im_internal
</Input>

<Input eventlog>
    Module im_msvistalog
</Input>

# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
  Module xm_csv
  Fields     $date,  $time,  $s-sitename, $s-computername, $s-ip,  $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip,  $cs-version, $csUser-Agent, $cs-Cookie, $cs-Referer,  $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes, $time-taken
  FieldTypes string, string, string,      string,          string, string,     string,       string,        integer, string,       string, string,      string,        string,     string,       string,   integer,    integer,       integer,          integer,   integer,   integer
  Delimiter ' '
  QuoteChar '"'
  EscapeControl FALSE
  UndefValue -
</Extension>


# Convert the IIS logs to JSON and use the original event time
<Extension json>
    Module      xm_json
</Extension>

<Input iis>
    Module    im_file
    File      "C:\\inetpub\\logs\\LogFiles\\W3SVC\\u_ex*"
    
    SavePos  TRUE

    Exec if $raw_event =~/^#/ drop();\
       else\
       {\
        w3c->parse_csv();\
        $EventTime = parsedate($date + " " + $time);\
		$EventTime = parsedate($date + " " + $time + "Z");\
        $SourceName = "IIS";\
		$raw_event = to_json();\
		$Message = to_json();\
       }
</Input>

<Output graylog>
    Module      om_udp
    Host        glog.wa.local
    Port        12201
    OutputType  GELF

	Exec file_write("C:\\nxlog\\data\\nxlog_output.log",  $Message);
</Output>

<Route 1>
    Path eventlog => graylog
</Route>

<Route 2>
    Path internal => graylog
</Route>

<Route 3>
    Path iis => graylog
</route>

Nxlog is installed to C:\Nxlog, nxlog.log show neuther warning nor error messages, just writes that nxlog started.

Also debug output to nxlog_output.log works just fine - it’s full of json messages. But graylog shows no one message from IIS.

What can I check else to find out the reason?


(Tess) #2

To verify:

  • When you query Graylog for the source host, it does show messages from the WinEventLog?
  • When you inspect the GELF receiver on 12201, and you tail its incoming messages, do you find any hints there?
  • Are the IIS logs actually getting updated? Is there actual traffic going in there? :smiley:
  • What happens if you manually trigger an update to those files; just add a line manually that conforms to the expected format.

Just a few hunches here…


(Zloy) #3

Thank you for your hints, @Totally_Not_A_Robot!
I found out that log records were actually sent by nxlog, but were rejected by graylog due to parsing errors.

I suppose that parsing errors (time was expected to contain integer instead of strings) originate from message schema changes during my attempts to map fields into types.

Thank you again for your help! The issue is closed now.


(Beth) #4

I think I am experiencing the same problem. I have been using nxlog for two years and my IIS logs were being sent to Graylog without any problem. A few days ago I had kernel updates to my system (Debian) and rebooted. Since the reboot, I have received Windows logs without any problem, but IIS logs are being rejected by Graylog with the following messages:

{“type”:“mapper_parsing_exception”,“reason”:“failed to parse [time]”,“caused_by”:{“type”:“illegal_argument_exception”,“reason”:"Invalid format: “22:52:30” is malformed at “:52:30"”}}

The problem is I don’t know what to fix. Is this a problem with nxlog/IIS or with Graylog? Below is my nxlog config:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension gelf>
Module xm_gelf
</Extension>

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension json>
Module xm_json
</Extension>

<Extension fileop>
Module xm_fileop
</Extension>

<Extension w3c>
Module xm_csv
Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs(User-Agent), $cs(Referer), $sc-status, $sc-substatus, $sc-win32-status, $time-taken
FieldTypes string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
Delimiter ’ ’
QuoteChar ‘"’
EscapeControl FALSE
UndefValue -
</Extension>

<Input in>
Module im_msvistalog
</Input>

<Input IIS>
Module im_file
File “C:\inetpub\logs\LogFiles\\u_ex*”
SavePos TRUE

Exec if $raw_event =~ /^#/ drop();				\
   else							\
   {							\
        w3c->parse_csv();					\
        $EventTime = parsedate($date + " " + $time  + "Z");	\
        $SourceName = "IIS";				\
        $raw_event = to_json();				\
   }

</Input>

<Output sslout>
Module om_ssl
Host 10.20.x.x
Port 14209
CAFile %ROOT%\cert\mycert.cer
AllowUntrusted TRUE
OutputType GELF_TCP
</Output>

<Route 1>
Path in => sslout
</Route>

<Route 2>
Path IIS => sslout
</Route>

Thank you to anyone that can help.


(system) closed #5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.