Pipeline rules problem concatenating existing fields into a new field

Graylog Central (peer support)
1

Hello
Im trying to create a new field within IIS logs for a username / ip combination using pipelines.
So far I can get a new field to appear in the logs, but it contains the literal field name not the actual field values I was expecting.
Ive read all the the posts in this forum that I could find that looked similar, but have not found a solution yet.

Here are the details:
IIS logs are sent from a windows server via nxlog using the w3c extension

<Extension w3c>
    Module 			xm_csv
    Fields 			$date,  $time,  $s-ip,  $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip,  $csUser-Agent, $cs-Referer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken $X-Forwarded-For
    FieldTypes 		string, string, string, string,     string,       string,        integer, string,       string, string,        string,      integer,    integer,       integer,          integer,            string
    Delimiter 		' '
    EscapeChar 		'"'
    QuoteChar 		'"'
    EscapeControl 	FALSE
    UndefValue 		-
</Extension>

With

OutputType GELF

In graylog, the processing order is like this:

1|AWS Instance Name Lookup  |active
2|GeoIP Resolver            |active
3|Message Filter Chain      |active
4|Pipeline Processor        |active

The pipeline rule looks like this:

rule "user and ip"
when 
   has_field("c-ip") AND has_field("cs-username")
then
   let userandip = concat(to_string("$message.cs-username"), (to_string("$message.c-ip")));
   set_field("mynewfieldforuserandip", userandip);
end

The resulting log now has the field ‘mynewfieldforuserandip’, but its value is:

$message.cs-username$message.c-ip

both fields ‘c-ip’ and ‘cs-username’ exist in the log message.

Ive tried different variations in this rule, including just setting one field to another rather than concat, but same result.

Does anyone know how this should be done?
Thanks

2

remove quotes in the to_string()…

let userandip = concat(to_string($message.cs-username), (to_string($message.c-ip)));

But the “-” are still causing you issues. so re quote them like this:

let userandip = concat(to_string($message.`cs-username`), (to_string($message.`c-ip`)));

That is the ` and not a ’ around the field names.

1 Like
3

Thanks very much for replying, you were absolutely right, it was the “-” causing me the issue. I had tried without quotes for the whole ‘$message…’ part but I had not idea that backticks were needed to quote a hyphen.
I removed the quotes and added backticks to the field names, and now it works as expected. Brilliant!

Many thanks

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.