Pipeline regex arrya for mac addresses don't work

[kalachev]

Hello all.
Graylog 4.3.13.
I write simple pipeline to exctract mac addresses from messages like this “An ip address conflict is detected. 00:00:00:00:00:01 and 00:00:00:00:00:00 share the same IP address 10.0.0.1”
Log get from Syslog UPD input.
Pipeline:

rule "Zywall find arp message with macs"
when
contains(to_string($message.message), "An ip address conflict is detected", true)
then
let ip_arp = to_string($message.zw_arp_ip);
let mac = regex("(([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2}))", to_string($message.message));
let mac1 = to_string(mac["0"]);
let mac2 = to_string(mac["1"]);
set_field("action", "arp attack");
set_field("mac1", mac1);
set_field("mac2", mac2);
set_field("ts_hour", $message.timestamp.hourOfDay);
set_field("ts_minute", $message.timestamp.minuteOfHour);
set_field("ts_second", $message.timestamp.secondOfMinute);
let ts = to_string($message.timestamp);
set_field("ts", ts);
end

Then i simulate this pipeline i get

action
arp attack
mac1
00:00:00:00:00:01
mac2
00:
ts
2023-06-01T22:06:51.150Z

why mac2 is 00, not the 00:00:00:00:00:00?

if i try convert regex var to string via

let test = to_string(regex("(([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2}))", to_string($message.message)));
set_field("test", test);

i got test field {0=00:00:00:00:00:01, 1=00:, 2=01}. Array has not 2nd mac, only 3 members:
1 mac, 5th & 6th group of 1 mac address.
Can anybody help me with this?

[gsmith]

Hey @kalachev

What do you get when using this, just curious

(\w{2}:\w{2}:\w{2}:\w{2}:\w{2}:\w{2})

It seams to test out good.

[kalachev]

In this situation it find only 1 mac
let test = to_string(regex(“(\w{2}:\w{2}:\w{2}:\w{2}:\w{2}:\w{2})”, to_string($message.message)));
let ip_arp = to_string($message.zw_arp_ip);
let mac = regex(“(\w{2}:\w{2}:\w{2}:\w{2}:\w{2}:\w{2})”, to_string($message.message));

Added fields

mac1
00:00:00:00:00:01
test
{0=00:00:00:00:00:01}

In pipilene 2 symbols of \ , i don why forum change 2 \ symbols for 1 \ symbol

Same situation with regex (.{2}:.{2}:.{2}:.{2}:.{2}:.{2})

[gsmith]

Hey,

I forgot to mention, in the pipeline they have to be double escaped.

let ipaddress = regex("(x-forwarded-for\\:\\s*(\\S+))", to_string($message.message));

[gsmith]

@kalachev

If that doesnt work , can you show you full pipe?

[kalachev]

@gsmith
Yes, i escaped them, and it didn’t help.
What do you mean “full pipe”?
If you mean all pipeline, that its empty, only 1 stage with 1 rule from 1st message.

[gsmith]

@gsmith

can you show you full pipe?

Full Pipeline Rule

rule "Between 6 AM and 6 PM"
when
	( to_long(to_date($message.timestamp, "American/Chicago").hourOfDay) >= 0 AND to_long(to_date($message.timestamp, "American/Chicago").hourOfDay) <= 6 ) OR
	( to_long(to_date($message.timestamp, "American/Chicago").hourOfDay) >= 18 AND to_long(to_date($message.timestamp, "American/Chicago").hourOfDay) <= 0 )
then
	set_field("trigger_workhours_off", true);
end

[kalachev]

Here

rule "Zywall find arp message with macs"
when
contains(to_string($message.message), "An ip address conflict is detected", true)
then
//let test = to_string(regex("(([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2}))", to_string($message.message)));
//let test = to_string(regex("(\\w{2}:\\w{2}:\\w{2}:\\w{2}:\\w{2}:\\w{2})", to_string($message.message)));
let test = to_string(regex("(.{2}:.{2}:.{2}:.{2}:.{2}:.{2})", to_string($message.message)));
set_field("test", test);
let ip_arp = to_string($message.zw_arp_ip);
let mac = regex("(([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2}))", to_string($message.message));
let mac1 = to_string(mac["0"]);
let mac2 = to_string(mac["1"]);
let new_message = concat("ARP on ip ", ip_arp);
let new_message = concat(new_message, "
mac 1st device - ");
let new_message = concat(new_message, mac1);
let new_message = concat(new_message, "
mac 2nd device - ");
let new_message = concat(new_message, mac2);
let new_message = concat(new_message, "
Need find problem device");
set_field("message", new_message);
set_field("action", "arp attack");
set_field("ts_hour", $message.timestamp.hourOfDay);
set_field("ts_minute", $message.timestamp.minuteOfHour);
set_field("ts_second", $message.timestamp.secondOfMinute);
let ts = to_string($message.timestamp);
set_field("ts", ts);
set_field("mac1", mac1);
set_field("mac2", mac2);
end

[gsmith]

Thx @kalachev

Sorry, Im getting tire so my typing is a little strange. Im going to plug this is and run it through my Lab.

EDIT haha I may have to clean up the russian a little bit.

[kalachev]

Thx @gsmith. I will be waiting result.

[gsmith]

Hey @kalachev

So i did some simple test using this.

rule "Extract multiple  fields"
when
  has_field("message")
then
   let mac = regex("(\\w{2}:\\w{2}:\\w{2}:\\w{2}:\\w{2}:\\w{2})", to_string($message.message));
   set_field ("mac1",mac["0"]);
   set_field ("mac2",mac["1"]);    

end

My results were the same, I would need to test more later, or perhaps someone else here would know a better way of executing this.

EDIT: i event tried to use GROK , same results only one came through, I know im missing something but not sure what.

[kalachev]

@gsmith
But regex_replace work correctly, it change both macs

let test_regex = to_string(regex_replace("(.{2}:.{2}:.{2}:.{2}:.{2}:.{2})", to_string($message.message), "test"));
set_field("test_regex", test_regex);

Result: An ip address conflict is detected. test and test share the same IP address 10.0.0.1

[kalachev]

So anybody got some idea, why regex don’t work as expected?