Graylog Pipeline rule with IPV6 not working

1. Describe your incident:
ipv6 pipeline rule not work on field type IP.

2. Describe your environment:

  • OS Information: docker

  • Package Version: 6.1.4

  • Service logs, configurations, and environment variables:

Configured two pipeline rules because of trying to solve my issue.
my issue is tag logs with “ipv6 tag” and normalize the IPV6 addresses to lowercase.

The first pipeline rule to check if grok works with pattern IPV4:

rule “fp_classify_ipv4”
when
grok(value: to_string($message.fp_Src), pattern: “%{IPV4}”).matches == true
then
// add tag ipv4
set_field(
field : “tags”,
value : “ipv4”
);
end

this pipeline rule works great.

Second Rule for IPV6:

rule “fp_classify_ipv6”
when
grok(value: to_string($message.fp_Src), pattern: “%{IPV6}”).matches == true
then
let my_src_ipv6 = lowercase(to_string($message.fp_Src));
set_field(“fp_Src”, to_ip(my_src_ipv6));

// to check if it works with other field type
set_field("fp_Src_ipv6_lc", my_src_ipv6);

// add tag ipv6
set_field(
    field : "tags",
    value : "ipv6"
);

end

This pipeline rule matches nothing and i dont know why.

The field type fp_Src is IP, because we use cidr match for searching etc.

example values are:
fp_Src: 1.1.1.1
fp_Src: 2A00:1450:400C:C00::5E

3. What steps have you already taken to try and solve the problem?

create multiple pipeline rules to get it work, but nothing worked. One of them are here posted.

4. How can the community help?

let me know why my pipeline rule for ipv6 not matches for a field with the correct content.

Thanks for your help.

I’m guessing the pattern doesn’t know how to handle the :: omissions.

Graylog ships a pattern for ipv6 and it can handle :: .

Here the pattern, copied from grok patterns section in graylog:

((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?
1 Like

i tested this out with grok debugger and it works.
https://grokdebugger.com/

I used this rule for testing:

rule "zz"
when
  true
then
  let fpSrc = "2A00:1450:400C:C00::5E";
  let result = grok(value: fpSrc, pattern: "%{IPV6}");
  set_field("aa",result);
end

And got this result:

aa
{"IPV6":"2A00:1450:400C:C00::5E"}

So the basic grok matching is working.

thanks for testing, can you do one more test if this works when the field type is ip and the when is not a hardcoded with true statement?

like my example above?

thanks for testing and help.

I don’t have any IP data in my test logs, but it tests out successfully with IP data in the rule.
To further debug, I would change your rule condition to true and then see what the grok().matches is actually returning. I suspect we are never actually executing the rule body.

rule “zz”
when
true
then
let fp_str = “2A00:1450:400C:C00::5E”;
let fp_ip = to_ip(fp_str);
set_field(“aa_ip”, fp_ip);

let result = grok(value: to_string($message.aa_ip), pattern: “%{IPV6}”);
set_field(“aa”,result);

let grokmatches = grok(value: to_string($message.aa_ip), pattern: “%{IPV6}”).matches;
set_field(“aa_matches”, grokmatches);
end

aa
{“IPV6”:“2a00:1450:400c:c00::5e”}
aa_ip
2a00:1450:400c:c00::5e
aa_matches
true

sorry, in pipelines i have added for processing a new stage and in previous stage to get to the correct stage there was wrong condition to process the ipv6 classify rule. My fault, i am appreciate your help. Thank you problem is now solved

1 Like

Glad you were able to resolve it.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.