Hey there, I was wondering if is there any way of having a kind of transient states across the pipeline execution.
Here is a typical use case (keep in mind I’m trying to push metrics into graylog):
- Identify when a specific field in a log changes from OK (based on the value of the field of a previous log) to WARN (based on the value of the field in the current log) and based on that trigger alerts.
- Aggregate logs and save space into ES by getting multiple logs incoming into the Stream and saving only diffs… this avoid the system to save every and each repeated and identical log entry into ES.
I’m checking that because currently we are using Riemann as an event stream but Riemann does not provide HA or scale.
It would be awesome to have that in the pipeline processing.