Right now this all just appears in the ‘message’ field. I plan on using a pipeline rather than an extractor to separate out the fields (as not all incoming logs to that input will necessarily be in the same format).
Is there an easy way using pipeline rules to break the contents of the message out into separate fields?
rule "get_key_value_out"
when
// add some more to get this run only
// on the messages where it can run
has_field("message")
then
// this will extract the key-value
// writes it to fields with a prefix
set_fields(
fields:
key_value(
value: to_string($message.message),
trim_value_chars: "\ ",
delimiters: ",",
kv_delimiters: ":"
),
prefix: "kv_"
);
end
That was extremely helpful, thank you. Quick follow on query - my key value separator is actually ": " (colon whitespace). The event_time fields contains colons in the data value e.g. 12:00:00.
I have tried
kv_delimiters: ": "
And
kv_delimiters: ":\ "
The first just ignores the whitespace, and breaks the event_time and the second seems to be an error.
Is there a way to get the kv_delimiter to be ": "?
