Receiving messages are stopping when running set_fields in pipeline

I have some troubles with using grok patterns in Graylog pipelines.
I have a simple test rule to parse message field:

rule "pipeline"
    let mess = to_string($message.message);
    let patt = grok(pattern:"%{GREEDYDATA:message2}",value: mess,only_named_captures: true);

It works correctly in Simulation, but when I including this rule into stage all messages are stopping to receive in Graylog.
When I just commented string set_fields(patt);, messages are going on correctly receiving.
It repeats with any trying to post grok results in field: set_field(“message2”,patt[“message2”]); etc.
But set_field(“pipeline”,“true”); - setting additional field with hardcoded params is working correctly.

Where is the problem?

Graylog Enterprise 4.2 (docker)

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

It appears the way you have your Grok set up, it will grab the entire message and put it into field message2 This is because your GROK: %{GREEDYDATA:message2} says to do exactly that.

You can use an online GROK Debugger to figure what your GROK would do.

For instance if you had the following message:

Commit job started processing. Dequeue time=2022/01/26 08:03:40. JobId=168.User: Bob. Commit Description: Remove DMZ NAT - RTZ34 plus tags

and you applied this GROK pattern:


You would get this result:

  "fw_admin": [
  "thing_that_happened": [
      " Remove DMZ NAT - RTZ34 plus tags"

Play around with GROK a bit and see if you can capture your message, if you need help, be sure to post a sample obfuscated message and the GROK you have progressed to…

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.