If I disable syslog on the device to simulate a lack of incoming messages, the alert does not fire.
If I click Replay search, the count(message)<1 returns this:
While retrieving data for this widget, the following error(s) occurred:
Elasticsearch exception [type=illegal_argument_exception, reason=Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [message] in order to load field data by uninverting the inverted index. Note that this can use significant memory.].
Hi Jeffrey,
I think the error is coming because you’ve selected “message - string” as the field for your count condition. Elasticsearch is objecting to being asked to to an aggregation operation on a text field, as that would be a very resource-heavy operation, so by default it won’t let you do that.
From your description, you’re only interested in the actual count of messages (as opposed to a count of fields called ‘message’, which is effectively what you’re asking it for now), so you could leave that dropdown empty and it will just count the number of messages.
But - even if you fix the error, I don’t believe it’s possible to do what you’re after from a simple alert. Graylog doesn’t have any kind of fixed inventory or device list, so it has no idea which sources it ‘should’ have heard from. At the moment you’re effectively saying “tell me which devices you haven’t heard from in the last 5 minutes”, but within this query Graylog has no context to decide which devices you’re expecting to have logged messages.
I can’t see any kind of easy workaround to achieve what you’re after, although I’m still quite new to Graylog so others may have a better suggestion. But this kind of ‘change over time’ query seems to be what the Graylog Correlation Engine is intended for, but does require an enterprise licence.
