AlertManager Callback Plugin

the NEW Marketplace Plug-Ins
1

AlertManager Callback Plugin

@StefanHufschmidt

Plugin 1.2.2

A plugin for Graylog which provides the possibility to send alerts to the Prometheus AlertManager API.

Build Status codecov

Graylog AlertManager Notification Plugin

This plugin can be used for connecting Graylog alerts to the Prometheus AlertManager.

The plugin development is based on Graylog2/graylog-plugin-sample which is mentioned in the Graylog plugin documentation.

Note: this plugin is currently only available as “Legacy Alarm Callbacks” item after adding it to Graylog

The current plugin version 1.2.2 has Graylog 3.3.5 as target. The plugin version 1.2.1 (see releases) targets Graylog 2.5.0 and above.

Use Case

You are using a Graylog for checking the logs for errors, a Prometheus for checking the service metrics and you would like to organize your alerts with a AlertManager you need to get your Graylog alerts into the AlertManager. This plugin provides the possibility to send your Graylog notifications with a AlertManager-Callback to your AlertManager.

AlertManager Endpoint

The plugin uses the /api/v1/alerts endpoint of AlertManager. You can find some documentation about this endpoint here.

Provided Information

The plugin provides the AlertManager several information out of the box:

  • stream_title - The title of the stream triggering the alert condition in Graylog
  • triggered_at - The time of triggering the alert condition in Graylog
  • triggered_rule_description - The generated rule description of triggered alert condition in Graylog
  • triggered_rule_title - The title of alert condition rule in Graylog

All of those information will be added as annotation.

The values startsAt , endsAt and generatorURL will be transmitted to the AlertManager as well. startsAt will be set to the point of time when the condition triggered the alert. endsAt will be set to the point of time when the condition triggered the alert plus the set grace time which is configured for the alert.

Additionally you can configure your own custom annotations and labels which should be submitted to the AlertManager (see screenshot below). You can use the JMTE Template as you might already know from the Graylog E-Mail Notification Callback.

List of provided keys you can use inside JMTE Templates:

  • stream_url - The stream url.
  • stream - The specific stream object. There you can use the properties of the stream object e.g. stream.title
  • alertCondition - The specific triggered alert condition. There you can use the properties of the alert condition oject e.g. alertCondition.createdAt
  • check_result - The specific check result. There you can use the properties of the check result object e.g. check_result.triggeredAt
  • backlog - A list containing messages matching the triggered condition if any. You can iterate through them with ${foreach backlog message}${message} ${end}
  • backlog_size - The amount of matching messages.

How to deploy on Graylog

You can easily build the plugin by executing ./gradlew build -x check --no-daemon . Afterwards there should be a .jar file inside the build/libs/ directory. Follow the instructions mentioned here to deploy this .jar file.

How to develop locally

You can use Docker and Docker-Compose to test changes of this plugin locally by following these steps:

  1. Build the JAR
  2. Run docker-compose up inside the development directory
  3. Access your local Graylog instance web-interface at http://localhost:9000/
  4. You can configure an alert to be send to http://alertmanager:9093/api/v1/alerts and access the AlertManager via http://localhost:9093/

Screenshots

Planned Features

  • Implementation of EventNotification for usage as non-legacy notification plugin

You would like to contribute anything? - Take a look at CONTRIBUTING.md.

License

See LICENSE

Copyright

Copyright (c) 2019-2020 G DATA CyberDefense AG and other authors.