Rename Multiple Fields Using Pipeline Rule

[libertasfox]

Greetings Grayloggers,

I’m really struggling to find a way to rename multiple field names. I’v scoured the support site but to no avail. I was wondering if anyone can point me in the right direction. I have a large list which I’ve reduced below, that I need to remove what essentially is a prefix from the field. I’m shipping this over via Beats from a Linux box using the Sidecar. It starts out as json so I’m using the “json.keys_under_root: true” and “json.add_error_key: true” to parse it prior to arriving in Graylog. When it arrives in Graylog, each line/field is populated with something like this:

remove_this_prefix_agent_red

I’ve used the pipeline code below but it only works on one line, not all.

---

rule rule “remove_this_prefix_ from fields”
when
has_field(“remove_this_prefix_red”)
then
rename_field(“remove_this_prefix_agent_red”, “red”);
end

What I’d like to be able to do is have a then statement that covers all of it but I’m not sure what code needs to go in “when” (see below). I’ve messed around with regex and other assorted ways to rename the field but I know I’m missing something.

---

rule “remove_this_prefix_ from fields”

when
What goes here???

then
rename_field(“remove_this_prefix_agent_red”, “red”);
rename_field(“remove_this_prefix_agent_blue”, “blue”);
rename_field(“remove_this_prefix_agent_green”, “green”);
rename_field(“remove_this_prefix_light_blue”, “light_blue”);

[megan201296 (Megan)]

@libertasfox have you tried putting in the when field something like the source of the logs (or the input, etc etc) that contains those fields? Something like $message.source == <source>. The when section doesn’t have to reference the prefix fields at all, it just has to return “true” for any logs containing those fields you want to rename.

[libertasfox]

Hi Megan,
Thank you!! I don’t know why I didn’t think of that but that worked perfectly! You saved my sanity rolling into this weekend. I really appreciate your help. Have a good one!

-Joe