Using Grok patterns in Email notification


(Lewis Quin) #1

I have been reading through various posts on using GROK patterns in extractors; and I have managed to create the pattern needed, however I am unable to find how to reference the pattern in the email notification??


(Jochen) #2

Why do you want to reference a Grok pattern in an email notification?

Maybe you want to include a message field in the emails?
That’s described in the documentation: http://docs.graylog.org/en/2.3/pages/streams/alerts.html#email-alert-notification


(Lewis Quin) #3

You are probably correct, this is all new to me… I have read through the documentation but there is a certain amount I probably don’t completely understand. Basically what I want to do/have done is as follows:

I have defined a GROK pattern that extracts a source IP address from a syslog message
I have defined a GROK pattern that extracts a destination IP address from a syslog message.

What I want to know is the syntax for using these values within the email template, for example:

{if backlog} {foreach backlog message}
[source_ip_listed_here]
[destination_ip_listed_here]

If I use a regular expression I see I can save it as a “field” but with GROK I do not see such an option?


(Jochen) #4

What’s the name of the fields you’ve extracted these IP addresses into?
What’s the Grok pattern you’re using?


(Lewis Quin) #5

There is an 'inside" IP address, and an ‘outside’ IP address in the Syslog message which I am calling using the following

inside:%{IP}
outside:%{IP}

they return the appropriate values when I use the 'try’button


(Jochen) #6

You should probably assign a destination field to these patterns. :wink:

See http://docs.graylog.org/en/2.3/pages/extractors.html#using-grok-patterns-to-extract-data for some examples.


(Lewis Quin) #7

Thanks, is the destination field the word after the “:”?

i.e.

outside:%{IP:dst_ip}

Where “dst_ip” would be the field?, if so I tried this (below) which did not work?

{if backlog} {foreach backlog message}
${message.fields.dst_ip}


(Jochen) #8

Yes, correct.

What was the complete template?
What did you expect it to contain?
What messages triggered the alert?


(Lewis Quin) #9

It was a syslog message that fired (%ASA-4-106023) which shows a packet being denied from one IP (inside) to another (outside). I guess I thought the field would contain the IP address that it obtained when testing the GROK pattern

${if backlog} 
${foreach backlog message} 
Destination IP: ${message.fields.dst_ip}
Received: ${check_result.triggeredAt}
Event time: ${message.fields.EventReceivedTime} 
${end}${else}<No backlog> ${end}

##########
Alert Description: ${check_result.resultDescription}
Date: ${check_result.triggeredAt}

Notification ID: ${stream.id}
Event: ${stream.title}

Stream description: ${stream.description}
Alert Condition Title: ${alertCondition.title}
{end}

Triggered condition: ${check_result.triggeredCondition}
##########

${if backlog}Last messages accounting for this alert:
${foreach backlog message}${message}

${end}${else}<No backlog>
${end}

(Jochen) #10

What was the exact message which should have triggered the alert and what was the content of the notification email?


(Lewis Quin) #11
Destination IP: 
Received: 2017-10-16T14:33:20.876Z


##########
Alert Description: Stream received messages matching <message:"106023"> (Current grace time: 0 minutes)
Date: 2017-10-16T14:33:20.876Z

Notification ID: 59e2a244f0db100a20678390
Event: Blocked Outbound Traffic

Stream description: Traffic that is denied by ACL Alert Condition Title: Blocked Traffic {end}

Triggered condition: 0f98c24e-80f3-4c94-8035-677a7bd19a84:field_content_value={field: message, value: 106023}, stream:={59e2a244f0db100a20678390: "Blocked Outbound Traffic"} ##########

Last messages accounting for this alert:
source: %ASA-4-106023: | message: %ASA-4-106023: Deny udp src inside:10.100.52.11/389 dst outside:10.140.10.58/56302 by access-group "inside_access_in" [0xbe9efe96, 0x0] { from_syslog: true | level: 4 | gl2_remote_ip: 10.100.52.10 | gl2_remote_port: 514 | IP: 10.100.52.11 | IPV4: 10.140.10.58 | streams: [59e2a244f0db100a20678390] | gl2_source_input: 59e0f67af0db1006381e529c | dst_ip: 10.140.10.58 | gl2_source_node: 42e0a79c-c621-4542-8085-e43e4d53a2a1 | _id: f4a50b91-b27e-11e7-b658-000c29f1c02d | facility: local4 | timestamp: 2017-10-16T14:33:19.817Z }

(Lewis Quin) #12
%ASA-4-106023: Deny udp src inside:10.100.52.11/389 dst outside:10.140.10.58/56302 by access-group "inside_access_in" [0xbe9efe96, 0x0]

(Jochen) #13

Are you sure that this is the same template?
It’s missing the line with “Event time”, which should be part of the generated notification even if the referenced message field doesn’t exist.


(Lewis Quin) #14
message ID: ${stream.id}

Event Information
===============
Event:Traffic blocked from leaving internal network
Detection IP: 10.100.52.10 
Source IP: ${message.fields.fields.dst_ip}
Destination IP: ${message.fields.dst_ip}
%{IP:dst_ip}
Port:
Received: ${check_result.triggeredAt}
%{DATA:dst_ip;string}
$%{DATA:dst_ip;string}

${if backlog} 
${foreach backlog message} 
$outside:%{IP}
$inside:%{IP}
Port: ${message.fields.dst_prt}
Event time: ${message.fields.EventReceivedTime} 
${end}${else}<No backlog> ${end}

Date: 



##########
Alert Description: ${check_result.resultDescription}
Date: ${check_result.triggeredAt}

Notification ID: ${stream.id}
Event: ${stream.title}

Stream description: ${stream.description}
Alert Condition Title: ${alertCondition.title}
{end}

Triggered condition: ${check_result.triggeredCondition}
##########

${if backlog}Last messages accounting for this alert:
${foreach backlog message}${message}

${end}${else}<No backlog>
${end}

(Jochen) #15

What does that mean?
Maybe you could add a little bit of context to the things you’re posting…


(Lewis Quin) #16

that was the exact template


(Jochen) #17

Okay, then what is %{IP:dst_ip} supposed to contain for example?
You also don’t iterate over the backlog variable to access each message but try to access ${message.fields.dst_ip} directly, which doesn’t work because there’s no message object in the template (only a collection of messages in backlog).


(Lewis Quin) #18

%{IP:dst_ip} <- was a failed attempt to get this to work. Not sure what you mean by the rest of the comment


(Jochen) #19

You cannot access ${message.fields.src_ip} without a {foreach} block around it.


(Lewis Quin) #20

Could you show me an example?
Thank you for your help