Alert processing query


(Robp1234) #1

Hi, I am using a collector to read from a logfile. All is working perfectly. The events as received into graylog look like:

eventtime: 2018-07-01T16:25:08Z, account_id: 123456, email: rob@madeup.com, ip_addr: 123.12.12.12 etc

Right now this all just appears in the ‘message’ field. I plan on using a pipeline rather than an extractor to separate out the fields (as not all incoming logs to that input will necessarily be in the same format).

Is there an easy way using pipeline rules to break the contents of the message out into separate fields?

Graylog version 2.4.5

thanks

Rob


(Jan Doberstein) #2

what you have given looks like a key-value pair each seperated by , and key and value seperated by :

so following the docs ( http://docs.graylog.org/en/2.4/pages/pipelines/functions.html#key-value ) you would have something like:

rule "get_key_value_out"
when
	// add some more to get this run only
	// on the messages where it can run
	has_field("message")
then
	// this will extract the key-value 
	// writes it to fields with a prefix 
   set_fields(
                fields: 
                        key_value(
                            value: to_string($message.message), 
                            trim_value_chars: "\ ",
                            delimiters: ",",
                            kv_delimiters: ":"
                            ),
                prefix: "kv_"
            );
end

(Robp1234) #3

That was extremely helpful, thank you. Quick follow on query - my key value separator is actually ": " (colon whitespace). The event_time fields contains colons in the data value e.g. 12:00:00.

I have tried
kv_delimiters: ": "
And
kv_delimiters: ":\ "

The first just ignores the whitespace, and breaks the event_time and the second seems to be an error.

Is there a way to get the kv_delimiter to be ": "?

thanks

Rob


(Jan Doberstein) #4

He Rob,

that might be a bug - from my current point of view. Do you mind opening a bug issue over at github

thank you


(system) #5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.