I’m trying to set up an alert for high average CPU utilization via Metricbeat logs.
I’m using a filter & aggregation as the event condition, which works as expected:
query: metricbeat_system_cpu_total_pct:>=0.50
group by: source
if avg() of metricbeat_system_cpu_total_pct >= 0.90
However, try as I might, I can’t find a way to actual report the CPU utilization that triggered the event in the email alert itself.
I’ve tried using a custom field, but since I can’t access the message itself this way i’m unable to get the total utilization as reported by the message. The only custom field I have that works is the hostname (${source.source})
I’ve also tried using check_result.triggeredcondition in the notification, but check_result appears to be completely empty in the resulting emails.
Is there some way to access the result of the aggregation in a notification?
I want the result of the aggregation (the average of the metricbeat fields) to appear in email notifications generated by this event. How do I do this?
The corresponding notification is below:
— [Event Definition] ---------------------------
Title: {event_definition_title}
Description: {event_definition_description}
Type: {event_definition_type}
--- [Event] --------------------------------------
Timestamp: {event.timestamp}
Source: {event.source}
Priority: {event.priority}
Triggered condition: {check_result.triggeredCondition}
Fields:
{foreach event.fields field} {field.key}: {field.value}
{end}
{if backlog}
— [Backlog] ------------------------------------
Last messages accounting for this alert:
{foreach backlog message}
{end}