Send alert based on message content

[benbow (Charles)]

Hi All,

I’ve read (most of) the documentation but I’m a little new to graylog so I’m still learning what all of the best practices are. We’re a smaller to mid-size company and have recently deployed graylog because centralized logging sounded like a good idea. Now I’m to the point where I’m trying to get some use out of it.

Basically, I plan to send the syslog of a bunch of Linux servers to graylog. That part I know how to do. I have already set up a stream for a subset of task-specific hosts. Now lets say I want to get an email every time a particular message string shows up in the logs for those hosts. Lets use this one as an example:

unregister_netdevice: waiting for lo to become free. Usage count = 1

How do I alert on exactly that string? The UI/docs are not completely clear on this. I thought I would add an alert condition where I would be able to specify the field as message, “value contains” as the condition, and the log message above as the value. But there’s no option for this. Reading the docs, it sounds like multiple words should not be used in alerts when the field is message because they are broken up into separate words it will match on log entries that you don’t expect. The docs do not say what to do when you want to alert on a string. I thought this would be the simplest possible example for setting up an alert in graylog but that turns out not to be the case.

What’s the right way to do this?

Thanks,
Charles

[Shane]

Are you wanting to alert on that entire string? You can create a stream that has multiple rules and just have each rule state something like “Contains unregister_netdevice” and “waiting” etc… or you could break apart that message into fields, such as create an extractor or pipeline for Usage count, and another field for “unregister_netdevice”. Then create a condition for that stream that only triggers if “Usage count =1” because essentially all messages would == 1 if they were in that specific stream. Then you apply an alert to that specific condition.

[jan (Jan Doberstein)]

Hej @benbow

i personal would recommend to create a pipeline or extractor that gets the interesting information out of the message field and create (single or multiple) new fields that contains only single parts and on that you can alert.

The important part is to process the message and modify/create fields with wanted Information. Because the processing will be only done once. If you have an alert that is searching for a regex that regex search is done on every minute on every available message.

That would eat more resources than just one split/modify of the message on ingest.

/jd