Slow processing of iis log inputs using parse_csv

Graylog Central (peer support)
1

I’m experiencing a slow iis log message processing rate when using the ‘w3c->parse_csv()’ method.
When processing log messages not requiring the method I’m getting 1000’s of messages a second being processed but using the method I’m down to 10’s of messages a second.
Is there any way of speeding up the process ?

Regards,

Harry W.

2

Hej @harryw

did you use nxlog to parse that log file?

How did you configure that?

3

Yes, I use ‘nxlog’ to parse the IIS logs.

Regards,

Harry W.

4

How did you configure that?

you forget my second question …

… and you are aware that you are asking the question how to speed up nxlog in the Graylog Community or?

5

Sorry about that.
The ‘NXLOG Inputs’ setup is as follows;

Exec $size = size($raw_event);
Exec if $raw_event =~ /^#/ drop(); \
else \
{ \
w3c->parse_csv(); \
$EventTime = parsedate($date + " " + $time); \
$EventTime = strftime($EventTime, "%Y-%m-%dT%H:%M:%SZ"); \
$sourcetype = "bet_disputes"; \
}

The ‘NXLOG Snippets’ setup is as follows;

{{if .Windows}}
<Extension w3c>
    Module xm_csv
    Fields $date, $time, $s-sitename, $s-computername, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs-version, $csUser-Agent, $csCookie, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes, $time-taken, $x-forwarded-for
    FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, string
    Delimiter ' '
    QuoteChar '"'
    EscapeControl FALSE
    UndefValue -
</Extension>
{{end}}