Pipllines and the grok-function

I am trying to use the grok-function in a pipline. But I’m failing. I do not mean the Grok-function in the Inputs, my plan is to use pipelines to extract data with Groks as described here.

2. Describe your environment:

  • OS Information:
    ubuntu 20.04

  • Package Version:
    Graylog 4.2.5

A little more in the Problem:

I have messages running into my steam to debug. The messages are ingested as Syslog on port 6666. To make it as simple as I can use this little bash-oneliner:

echo "testword1 testword2" | nc my-graylog.internal.network 6666

To extract both words I’m using this PoC-Rule:

rule "howto Grok in Pipeline"
when
has_field("message")
then
  let val = grok(
    pattern:"%{WORD:part_1) %{WORD:part_2)",
    value:to_string($message.message));
  set_field(
    field:"part_1",
    value:to_string(val["part_1"])
  );
  set_field(
    field:"part_2",
    value:to_string(val["part_2"])
  );
end

In the message I can see this error:

gl2_processing_error
    In call to function 'grok' at 5:12 an exception was thrown: Illegal repetition near index 0
    %{WORD:part_1) %{WORD:part_2)
    ^

My expectation would be to have two new fields, but I don’t. Where is my error?

Hello,
I might be able to help but I’m not that great in pipelines.
From what I seeing in your error…

I think it maybe something with this let val = From what I glanced at other posts perhaps it suppose to be just

grok( pattern:"%{WORD:part_1) %{WORD:part_2)", value:to_string($message.message));

Have after looking at this maybe a comma between…

pattern:"%{WORD:part_1), %{WORD:part_2)",

So I looked here and was unable to see this function let val=

https://docs.graylog.org/docs/functions

When you created this pipeline/rule you can test it out in the Simulator. Have you tried that? Then you can change the results if you wish.

I did a quick search for this situation maybe some of this posts perhaps have hints within them that may help.

After reading back and forth I found my mistake:

If I would close the Patterns also with a “}” it’s working like a charm. :face_with_hand_over_mouth:

I’ll close this thread.

1 Like

here an even nicer pipelinefunction to get it working:

rule "howto Grok in Pipeline"
when
  has_field("message")
then
  set_fields(
    grok(
      pattern:"%{WORD:part_1} %{WORD:part_2}",
      value:to_string($message.message)
    )
  );
end
2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.