Hello, Graylog community,
After doing a lot of research and testing, I decided to come here and ask if anyone could help me to shed some light on a particular situation.
I’m trying to configure a pipeline rule to extract some fields inside an original field called “message”, but for a reason, but for some unknown reason, this pipeline rule was not able to perform this simple task.
Let me explain better, I’m extracting information from a database and using Graylog sidecar/filebeat to collect these events, and the field message, contain some key-value pairs (as shown in the image down below) that contain useful information for my later searchable purposes.
I’m using the following pipeline rule, I found this code in many posts here and in other blogs, the most awkward for me it’s the simulation works as you can see in the following picture, but when I applied the rule and generate some logs/events the fields are not extracted.
rule "Safetica_Message_Fields"
when
has_field("message")
then
set_fields(
fields:
key_value(
value: to_string($message.message),
trim_value_chars: "",
trim_key_chars:"",
delimiters:"|",
kv_delimiters:"="
)
);
end
I feel that I’m doing something wrong, but I revisited all the configuration steps as the documentation says and as other blogs or tutorials say either, but I really cannot see what I’m doing wrong, if it’s possible, I would like your opinion about that.
Just one more observation, I changed the order of Message Processors Configuration but for me seems to not cause any effect, the next image shows the last configuration order I applied.
I appreciate in advance any comments.
Ps: before trying to use pipelines, I tried to use extractors with regexes, key=value extractors, and grok patterns extractors, none worked, the pipeline rule was the closest I came to achieving the correct field extraction, but if you guys have a different thought, please, share with me, I will be more than glad to test.