Extracting key="some value" in pipelines

Graylog Central (peer support)
1

Hi,
I’d like to extract fields from logs following the pattern key=value. Here’s a sample of the received messages:

time="2017-04-28T09:48:10+01:00" level=info msg="Starting alertmanager (version=0.5.1, branch=master, revision=0ea1cac51e6a620ec09d053f0484b97932b5c902)" source="main.go:101"

I’ve tried using the key_values function, however this function is not dealing properly with values quoted with spaces. Here’s a sample code/output:

let matches = key_value(to_string($message.message));
set_fields(matches);
-------------- OUTPUT------------
time  -> "2017-04-28T09:48:17+01:00"
msg -> "Starting
branch -> master,
source -> "main.go:101"

How can I extract fields from messages following the above pattern using pipelines? Fields values may be quoted and have spaces.

2

As you can see in the documentation the whitespace is the default k/v delimiter. You need to have a uniq delimiter. But your message does not provide one.

If you want to use k/v you need to adjust your logs or use other ways to extract the information from the log file.

3

It’s how Go apps log. Third-party apps can’t be adjusted. It would be helpful if the key_value function could be expanded to support quoted values.

4

Hej Charles,

feel free to open a feature request that describes the problem and what you would like to have.

regards
Jan

5

Done. Thanks for the link.