Create Pipeline for Event Definition

[gsmith (Greg Smith)]

All,
I would like to create an Event Definition for a message that was generated for a license renewal,not sure if I need to focus on a Pipeline/Extractor or if there is another way of accomplishing this. In this example I have a stream Called “License Expiring”, it has a rule called “Field EventID must match exactly 24040”. This collects a message every day around 6PM for 30 days (i.e. License Expires on the 30th day). What I would like to happen is to create alert/notification 5 days prior, then every day after that, before the 30 days is up. What I have tried with no success is to create a pipeline rule with something similar like this;

rule " License Expiring "
when
has_field(“message”) AND contains(“expire”,to_string($message.message))
then
set_field(“license_trigger”, true);
end

rule “5 Days”
// count 15 days
to_long(to_date($message.timestamp, “American/Chicago”).count_day) == 15
then
route_to_stream(name:“License Expiring In 5 Days”);
end

Unfortunately, this rule does not work and I’m not very good at creating pipelines yet.
If I could get something like this to work, I would route it to another Stream called “License Expiring In 5 Days”
Then I could create a notification/alert.

My Environment:
CentOS 7 Latest Version
Graylog 3.1.1+b39ee32
Elasticsearch-6.6.1-1.noarch
Mongodb-org-4.2.0

Researched;
PipeLine_Rules
PipeLine_Fuction
Function_Index

Any advice, Ideas or direction would be appreciated.
Thank you in advance.

[gsmith (Greg Smith)]

Tried a new Pipeline

rule " License Expiring "
when
has_field(“message”) AND contains(“expire”,to_string($message.message))
then
set_field(“license_trigger”, true);
end

rule “5 Days”
when
// count 15 days
(to_long(to_date($message.timestamp, “American/Chicago”).day_count) >= 15 AND
to_long(to_date($message.timestamp, “American/Chicago”).day_count) <= 30)
then
route_to_stream(name:"License Expiring In 5 Days”);
end

[jan (Jan Doberstein)]

how does the messages look like you want to run the rules on?

[gsmith (Greg Smith)]

@jan
Not good, I forgot to test out the rule on the Simulator.
Here is the message Im trying to use;

message
Your support contract is about to expire. Please renew it soon to keep your access to technical support and product updates.

And here is the Simulation trace

1 μs
Starting message processing
44 μs
Message eaa7b241-f76f-11e9-afa7-00155d601d11 running [Pipeline ‘Veeam’ (5db3662383d72e04d3db9e64)] for streams [000000000000000000000001]
104 μs
Enter Stage 0
173 μs
Evaluate Rule ’ License Expiring ’ (5db2670983d72e04d3d9a62c) in Pipeline ‘Veeam’ (5db3662383d72e04d3db9e64)
210 μs
Evaluation not satisfied Rule ’ License Expiring ’ (5db2670983d72e04d3d9a62c) in Pipeline ‘Veeam’ (5db3662383d72e04d3db9e64)
229 μs
Completed Stage 0 for Pipeline ‘Veeam’ (5db3662383d72e04d3db9e64), NOT continuing to next stage
244 μs
Exit Stage 0
256 μs
Finished message processing

Here is my Pipline with rules

[jan (Jan Doberstein)]

if the complete sentence is always the same - why not match on that exact content? It should speed up the processing and avoid false positive.

[gsmith (Greg Smith)]

@jan
I did some more reseach this weekend and found out that the software used in our environment starts logging “EventID 24040” 14 days prior to expiration date.
So I did some adjusting to pipeline replaced the field message with a EventID. I think this would be more precise. Now I’m piping the message with EventID 24040 to a different stream.

rule " License Expiring "
when
((to_string($message.EventID) == “24040”))
then
set_field(“license_trigger”, true);
end
Rule “Route License to Stream”
when
has_field(“license_trigger”)
then
route_to_stream(name:“License Expiring In 5 Days”);
end

Event Definition
Event Definition Name: License Evaluation Expiring

Edit Event Condition
Condition Type: Filter & Aggregation
Filter Search Query: EventID:24040
Create Events for Definition: If count () , EventID, Is >= Threshold 9 (9 would be 5 days before License Expires)

Notifications
Notification Settings Grace Period 24 Hours
Think I may have solved it. I’ll find out in 8 Days if everything works out.

[gsmith (Greg Smith)]

So far the pipeline is working as expected. Had to modify it a little.

rule " License Expiring "
when
((to_string($message.EventID) == “24040”))
then
set_field(“license_trigger”, true);
end
Rule “Route License to Stream”
when
has_field(field:“license_trigger”)
then
route_to_stream(id:“5db243bc83d72e04d3d960a2”);
end