Can someone help to write a pipeline rule to satisfy the below requirements?
If there is noIDS signature triggered from the specific source address to the specific destination address for the last 60 minutes, set the field alert=true
ex:
1.1.1.1 -------> 2.2.2.2 for signature A, trigger me an alert if its a first match for the last 60 mins.
1.1.1.1 -------> 2.2.2.2 for signature A, don’t trigger me an alert if its coming again within 60 mins.
this could easily be achieved by using a pipeline rule and an alert, but the condition of the lookup in the last 60 minutes is making it more difficult.
Are you able to install plugins on your Graylog installation? If yes, have a look at
this could help you to solve your problem.
Basic outline:
Combine Source Address and Destination Address into one field
Add field to message
Check if a prior message in the last 3600 seconds contains the same value in that combined variable
No -> This is an alert condition, set field alert = true
Yes -> done, this message is ok, set field alert = false
Then add an alert checking for alert=true yada yada yada
I’m busy right now, but I will try to write an example rule when I’ve got time
I really appreciate your quick turn-around. Thanks for your helping nature.
I wrote below pipeline rule based on your inputs. First rule is working as intended. But, I am stuck with second one. I can’t have the variables outside When condition. Also, how to implement last 3600 seconds condition in pipeline rules?
rule "Unique Info"
when
((has_field("dst") AND (to_string($message.dst) != "N/A")) AND (has_field("src") AND (to_string($message.src) != "N/A")))
then
let temp_field = concat(to_string($message.src), to_string($message.dst));
let combined_field = concat(to_string(temp_field), to_string($message.name));
set_field("unique_info",combined_field);
end
rule "Aggregated Alert"
let temp_field = concat(to_string($message.src), to_string($message.dst));
let combined_field = concat(to_string(temp_field), to_string($message.name));
when
((combined_field != $message.unique_info) AND minutes(60)) // (now()-3600)
then
set_field("crtical_alert",true);
end
I was busy this week with my last essay for my university, but it is finished now and I’ll have time for this now, sorry for the delay I’ll respond this evening, when I cooled down from this damn essay
here is my approach to your problem:
This needs billmurrin’s slookup function to be installed and working correctly.
The relevant messages need to be routed into an own stream for the slookup function to be able to query the right messages reliably.
Create unique_info field
Create a Pipeline and add this rule to the first stage:
rule "Unique Info"
when
((has_field("dst") AND (to_string($message.dst) != "N/A")) AND (has_field("src") AND (to_string($message.src) != "N/A")))
then
let temp = concat(to_string($message.src), to_string($message.dst));
let combined_field = concat(to_string(temp_field), to_string($message.name));
set_field("unique_info",combined_field);
end
Check for previous alerts with same unique_info
Create a second stage and add this rule to it:
rule "Aggregated Alert"
when
// Check if exists in last 60 Minutes
// slookup(StreamID, Source Field, Destination Field, Return Field, Relative Time, SortOrder)
$message.unique_info == slookup("your-stream-id-containing-this-messages", "unique_info", "unique_info", "unique_info", 3600, "desc")
then
set_field("crtical_alert",true);
end
Done
In theory, this should work. Now attach an alert to the stream to check wether critical_alert is true in a given time frame (I suggest 5 or so ) .
I hope this helps. I cannot verify these rules, since I do not have access to my system sadly, but I hope it works
I’ll respond this evening, when I cooled down from this damn essay 
