Pipeline rule, multiple values using regex function, only possible to return value ["0"]

[aguida79 (Alejandro Guida)]

Hi to everyone,

Working with the situation I mentioned in this post, https://community.graylog.org/t/pipeline-rule-to-generate-a-object-field-with-obtained-ips-multi-value-for-a-qname-from-a-dns-lookup-table, I found that is only possible to return the first match of a regex.

My pipeline testing rule:

rule “Regla Pipeline - Windows DNS Server - SilkService - Lookup DNS QNAME - MultiIP”
when
has_field(“QNAME”) AND NOT contains(to_string($message.QNAME), “empresa.com”)
then
let multiIPs = lookup(“DNS_QNAME_IP”, to_string($message.QNAME));
let multiIPs_extract = regex(pattern: “ipAddress=(.*?),”, value: to_string(multiIPs));
// Para Debug
set_field(“QNAME_IPs_Text”, to_string(multiIPs));
//
set_field(“QNAME_IPs_Test0”, multiIPs_extract[“0”]);
set_field(“QNAME_IPs_Test1”, multiIPs_extract[“1”]);
end

An Example:

Based on “QNAME_IPs_Text” content, exists “QNAME_IPs_Test0” with the first IP, but not exists “QNAME_IPs_Test1”, with the second IP showed in “QNAME_IPs_Text”.

To create my rule, I based the logic of my rule in the rule explained by @jan in Pipeline rule, multiple values using regex function return an empty table, so I’m think the logic it’s ok.

I’m not an expert in regex, so maybe my regex is not ok to catch up every IP address in “QNAME_IPs_Text” field, I don’t know. But in that case, it is strange that it can catch up the first ocurrence of an IP address, but no the others.

An additional question: anybody knows if it is possible to get all the values that the regex match (in my case, IP addresses), in the same field, separated by a comma, like an Object field? My final goal is that, to get all the resolved IPs of a DNS LookUp Table search in a single field.

Can anybody help me, or say to me what I’m doing grong?

Thanks in advance.

Regards,
Alejandro

[aguida79 (Alejandro Guida)]

As @tmacgbay mentioned in this other post, Pipeline rule to generate a Object field with obtained IPs (multi_value) for a QNAME, from a DNS LookUp Table, I use the debug() function to review the content of of the field
multiIPs_extract[“1”]
And again, like the other post, it’s empty, even should have content based on the example:

[root@neoappliance ~]# tail -f /var/log/graylog-server/server.log | grep "Dropped message from"
2020-04-13T15:04:09.505-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:09.505-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:09.505-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:09.506-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:10.467-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:10.468-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 
2020-04-13T15:04:10.468-03:00 INFO  [Function] PIPELINE DEBUG: Dropped message from 

So I’m think maybe there is a bug here, but I don’t know enough of Graylog to guarantee this.

Maybe any Devoloper of Graylog can take a look to this post and answer this last question.

Thanks in advance for all your help.

Regards,
Alejandro