IIS logs into graylog

Hi,
I have tried using nxlog on its own to get IIS logs into Graylog but that failed so I tried to bring SideCar into matters to get that to help but that seems to have only make it worse.

Please could someone explain or link to how I get IIS logs to appear in Graylog?

Thanks.

I have decided to go bac kto basic and have uninstalled both nxlog and sidecar.

i then reinstalled nxlog and can get half the config working. The windows events are fine, its the IIS which isnt.

Here is my nxlog.conf

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension syslog>
    Module xm_syslog
</Extension>

<Extension gelf>
    Module xm_gelf
</Extension>

# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.

<Extension w3c>
    Module xm_csv
    Fields $date, $time, $s-sitename, $s-computername, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs-version, $cs(User-Agent), $cs(Cookie), $cs(Referer), $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes $time-taken
    FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, string, string, string
    Delimiter ' '
    QuoteChar   '"'
    EscapeControl FALSE
    UndefValue  -
</Extension>

# Enable json extension
<Extension json>
    Module      xm_json
</Extension>
 
# Convert the IIS logs to JSON and use the original event time
# Uncomment IIS_IN section if logging for IIS logging

<Input eventlog>
     # Use 'im_mseventlog' for Windows XP, 2000 and 2003
    Module      im_msvistalog
    # Uncomment the following to collect specific event logs only
    Query       <QueryList>\
                    <Query Id="0">\
                        <Select Path="Application">*</Select>\
                        <Select Path="System">*</Select>\
                    </Query>\
                </QueryList> 
</Input>

<Input IIS_Site1>
    Module im_file
    File "L:\\logs\\LogFiles\\W3SVC2\\u_ex*.log"
    SavePos TRUE
    Exec if $raw_event =~ /^#/ drop();    \
        else    \
        {    \
            w3c->parse_csv();    \
            $EventTime = parsedate($date + " " + $time);    \
            $SourceName = "IIS";    \
            $Message = to_json();    \
        }
</Input>


<Output Event_Out>
    Module      om_udp
    Host        10.10.0.79
    Port        12201      
    OutputType  GELF
</Output>

<Output IIS_Out>
    Module      om_udp
    Host        10.10.0.79 
    Port        5555
	
</Output>


<Route IIS>
     Path IIS_Site1 => IIS_Out
</Route>

<Route Events>
    Path eventlog => Event_Out
</Route>

This is the input I have created using the RAW/PlainText UDP input type

Is this the right type of input?

Thanks.

You don’t need a separate input for the IIS logs, use the same as the Windows Event logs.
I route all my Windows Events and file logs using this one GELF UDP input.

Regards,

Harry W.

Also you can use the ‘nxlog snippet’ section to define the IIS log conversion …

{{if .Windows}}
<Extension w3c>
    Module xm_csv
    Fields $date, $time, $s-sitename, $s-computername, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs-version, $csUser-Agent, $csCookie, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes, $time-taken, $x-forwarded-for
    FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, string
    Delimiter ' '
    QuoteChar '"'
    EscapeControl FALSE
    UndefValue -
</Extension>
{{end}} 

That way all the nxlog configuration is in ‘graylog’ and you don’t have to configure the nxlog agent manually.

Thanks for teh help Harry, much apprecaited.

So now I have

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module      xm_gelf
</Extension>
 
<Input in>
    # Use 'im_mseventlog' for Windows XP and 2003
    Module      im_msvistalog
   Exec if ($EventID == 4202 or $EventID == 4208 or $EventID == 4302 or $EventID == 4304 or $EventID == 5004) drop();\
   else{\
        if ( $EventType == "INFO" ) $SyslogSeverityValue = 6;\
        if ( $EventType == "WARNING" ) $SyslogSeverityValue = 4;\
        if ( $EventType == "ERROR" ) $SyslogSeverityValue = 3;\
    }
</Input>

<Input IIS_Site1>
    Module im_file
    File "L:\\logs\\LogFiles\\W3SVC2\\u_ex*.log"
    SavePos TRUE
    Exec if $raw_event =~ /^#/ drop();    \
        else    \
        {    \
            w3c->parse_csv();    \
            $EventTime = parsedate($date + " " + $time);    \
            $SourceName = "IIS";    \
            $Message = to_json();    \
        }
</Input>
 
<Output out>
    Module      om_udp
    Host        10.10.0.79
    Port        12201
    OutputType  GELF
</Output>
 
<Route 1>
    Path        in, IIS_Site1 => out
</Route>

Where do I put in the code you suggested with the fields etc?

As you can probably tell, this is my first stab at graylog/nxlog so learning as I go.

Thanks.

Finally got it working! Thanks to all that helped me out.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
	Module 		xm_gelf
</Extension>

<Extension fileop>
	Module 		xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module 			xm_csv
    Fields 			$date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken
    FieldTypes 		string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
    Delimiter 		' '
    QuoteChar 		'"'
    EscapeControl 	FALSE
    UndefValue 		-
</Extension>

<Input iis>
    Module    	im_file
    File    	"L:\\logs\\LogFiles\\W3SVC2\\u_ex*"
    SavePos  	TRUE

	Exec		if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "IIS";    					 \
                    $Message = to_json();    					 \
                }
</Input>

<Input eventlog>
    Module      im_msvistalog
</Input>

<Output graylog>
    Module      om_udp
    Host        xxx
    Port        12201
    OutputType	GELF

    #Use the following line for debugging (uncomment the fileop extension above as well)
	#Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Route eventlog>
   Path        eventlog => graylog
</Route>

<Route iis-to-graylog>
	Path		iis => graylog
</Route>
1 Like

Thanks for share the solution!
I was having the same problem and now I have it working.
Cheers!

Hi Team i have done same what you proceed with configuration. But i am not able get the logs. I have done below steps and this is my config file for iis logs.

Step 1) Created the input for GELF_UDP with the port 12201
Step 2 )changes has done in config file as follows below.

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension gelf>
	Module xm_gelf
</Extension>

<Extension fileop>
	Module 		xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

#Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module 			xm_csv
    Fields 			$date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken
    FieldTypes 		string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
    Delimiter 		' '
    QuoteChar 		'"'
    EscapeControl 	FALSE
    UndefValue 		-
</Extension>

<Input iis>
    Module    	im_file
    File    	"L:\\logs\\LogFiles\\W3SVC2\\u_ex*"
    SavePos  	TRUE

	Exec		if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "IIS";    					 \
                    $Message = to_json();    					 \
                }
</Input>

<Input in>
    Module      im_msvistalog
# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
</Input>



#<Output out>
#    Module      om_tcp
#    Host        192.168.1.1
#    Port        514
#    Exec        to_syslog_snare();
#</Output>

<Output out>
	Module om_udp
	Host X.X.X.X
	Port 12201
	OutputType GELF_UDP
</Output>

<Route 1>
    Path        in => out
</Route>

<Route iis-to-graylog>
	Path		iis => graylog
</Route>

But i am not able to get the IIS Can anybody help on this.

Hi,

Can you reformat that into a code block, its super difficult to read and spot the error as it is currently.

Thanks

Hi mr_m_cox

Can you help how to do reformat that into a code block. I could not find it.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
	Module 		xm_gelf
</Extension>

<Extension fileop>
	Module 		xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module 			xm_csv
    Fields 			$date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken
    FieldTypes 		string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
    Delimiter 		' '
    QuoteChar 		'"'
    EscapeControl 	FALSE
    UndefValue 		-
</Extension>

<Input iis>
    Module    	im_file
    File    	"L:\\logs\\LogFiles\\W3SVC2\\u_ex*"
    SavePos  	TRUE

	Exec		if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "IIS";    					 \
                    $Message = to_json();    					 \
                }
</Input>

<Input eventlog>
    Module      im_msvistalog
</Input>

<Output graylog>
    Module      om_udp
    Host        xxxx
    Port        12201
    OutputType	GELF

    #Use the following line for debugging (uncomment the fileop extension above as well)
	#Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Route eventlog>
   Path        eventlog => graylog
</Route>

<Route iis-to-graylog>
	Path		iis => graylog
</Route>

My default logs path in C drive:/logs/w3cv/

Hi @grayloglearn

can you post your nxlog.log from the log folder? this might give you an insight what isn’t working and why it isn’t starting.

best regards,
theresa

Hi Micsnare,

Here is the nxlog.log

2017-06-07 11:20:54 INFO nxlog-ce-2.9.1716 started
2017-06-07 11:36:27 WARNING stopping nxlog service
2017-06-07 11:36:27 WARNING nxlog-ce received a termination request signal, exiting...
2017-06-07 11:36:28 ERROR module 'iis' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-07 11:36:28 ERROR module 'graylog' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-07 11:36:28 ERROR route iis-to-graylog is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-07 11:36:28 INFO nxlog-ce-2.9.1716 started
2017-06-08 17:05:55 WARNING stopping nxlog service
2017-06-08 17:05:55 WARNING nxlog-ce received a termination request signal, exiting...
2017-06-08 17:05:56 ERROR module 'iis' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-08 17:05:56 ERROR module 'graylog' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-08 17:05:56 ERROR route iis-to-graylog is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:64
2017-06-08 17:05:56 INFO nxlog-ce-2.9.1716 started

Hi,

I sent the nxlog.conf can you help me on this.

Hi team i am getting the below screen shot after installing the collector-side car in windows.
I have installed nxlog and sidecar in windows.

can anyone help onthis.

You dont need anything to do with sidecars to get your IIS log into Graylog, just NXlog will do the job.

Dear Matt,

Its good to hear with out sidecar we can get the IIS logs can you please help with step by procedure. I did what can i do but no luck. Can you please help where i did mistake. I have followed the below steps.

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension gelf>
	Module xm_gelf
</Extension>

<Extension fileop>
	Module 		xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

#Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module 			xm_csv
    Fields 			$date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken
    FieldTypes 		string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
    Delimiter 		' '
    QuoteChar 		'"'
    EscapeControl 	FALSE
    UndefValue 		-
</Extension>

<Input iis>
    Module    	im_file
   File    	"C:\\inetpub\\logs\\LogFiles\\u_ex*.log"
    SavePos  	TRUE

	Exec		if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "IIS";    					 \
                    $Message = to_json();    					 \
                }
</Input>

<Input in>
    Module      im_msvistalog
# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
</Input>



#<Output out>
#    Module      om_tcp
#    Host        192.168.1.1
#    Port        514
#    Exec        to_syslog_snare();
#</Output>

<Output out>
	Module om_udp
	Host X.X.X.X
	Port 12201
	OutputType GELF_UDP
</Output>

<Route 1>
    Path        in => out
</Route>

<Route iis-to-graylog>
	Path		iis => graylog
</Route>

Glad to have found this thread. I am close but not successful yet.
I have a Windows 2012 (test server) and want to get the logs into the system.

  1. I started over and downloaded a Ubuntu virtual machine as outlinned in the docs to run GrayLog. (it is running)
  2. I downloaded and installed NXLog onto my Windows Test Server, and started a service with pointer to my log files ‘u_ex*.log’ from the inside of my Windows Test Server.

I have not done anything with: SideCar, and cannot get a HTTP message to show up on the VM.

I am sure that I am close but not there .

If anyone sees something wrong with the setup I am attempting, or the information below I would welcome input.

Andy…

MyConf file from the TestServer

define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
	Module xm_gelf
</Extension>

<Extension fileop>
	Module xm_fileop
</Extension>

<Extension _json>
Module      xm_json
</Extension>

<Extension w3c>
Module xm_csv
Fields $date, $time, $s-sitename, $s-computername, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs-version, $cs(User-Agent), $cs(Cookie), $cs(Referer), $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes $time-taken
FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, string, string, string
Delimiter ' '
QuoteChar   '"'
EscapeControl FALSE
UndefValue  -
</Extension>

<Input eventlog>
	Module im_msvistalog
</Input>

<Input iis>
	Module im_file
	File "C:\\inetpub\\logs\\testFolder\\u_ex*.log"	
	SavePos TRUE
	 Exec if $raw_event =~ /^#/ drop();    \
    else    \
    {    \
        w3c->parse_csv();    \
        $EventTime = parsedate($date + " " + $time);    \
        $SourceName = "IIS";    \
        $Message = to_json();    \
    }
</Input>

<Output graylog>
Module      om_tcp
Host        xx.xx.xx.xx  #my Graylog VM
Port        12201
OutputType	GELF_TCP
</Output>

<Route eventlog>
	Path eventlog => graylog
</Route>

<Route iis-to-graylog>
	Path iis => graylog
</Route>

My Log from ‘nxlog.log’

2017-06-16 11:33:09 WARNING stopping nxlog service
2017-06-16 11:33:09 WARNING nxlog-ce received a termination request signal, exiting...
2017-06-16 11:33:10 INFO connecting to xx.xx.xx.xx:12201
2017-06-16 11:33:10 INFO nxlog-ce-2.9.1716 started

My Source on the instance of GrayLog I am running on the downloaded VM I am running.