Graylog pipeline keys with spaces

Does anyone know if graylog pipeline rule supports keys with spaces? Mine seems to completely skip the parsing.

set_fields (
        value: $message.message,
        delimiters: ",",
        kv_delimiters: ":",
        trim_value_chars: "",

e.g. test key:value

Hi @syntax

I don’t think so. Please post your real example messages, so we can help. Maybe replace space with something will work.



2021-06-19 14:23:15 Message details, file: c:\users\admin\desktop\file.exe, log type: alert, ip address:

i have about a thousand log formats…so adding a dash/underscore would be very tedious.

There is already a feature request out there to solve this that has been sitting around for a bit (since Jan 2020)- here is the original post I had on it that includes a link to the feature request to handle it:

Get all your friends to look at it and comment that it should be prioritized… :stuck_out_tongue:


@tmacgbay interesting. how did u manage to circumvent the issue?

I had a previous rule that broke out the event_description field as defined by a quoted section of the message, then broke out the portion that needed key_value() applied, cleaning up the spaces and commas. Note how the regex is non-capturing for the first two words, then once it’s done its work I am referencing indexes. Also note the commented out debug() functions so I could watch what this looked like as it went through. Not pretty and likely not efficient, but I didn’t have thousands of these coming in (small company)

    let e_message     = to_string($message.event_description);

    let desc_parts      = regex(pattern: "^(?:\\w+\\s+){2}(.*)\\.\\s+(.+)", value: e_message);
    set_field("event_action", to_string(desc_parts["0"]));
    let desc_lowered    = replace(lowercase(to_string(desc_parts["1"]))," , ", ", ");    //might have extranious comma's
    let desc_cleaned    = regex_replace("\\b\\s+", desc_lowered , "_");                 //replace unwonted spaces 
    let keyed_up        = key_value(desc_cleaned,
                                " ",
                                " "

    //debug("$$$$---Event to be :");

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.