Pipeline, Log Delay


(-) #1

HI Guys,

I would like to create a pipeline rule, for the log delay. (without Stream alert)

For example
If a the source not send log more than 3 day based on Stream or gl2_remote_ip , the piplene create a field “The host is down”.

Any idea?

Thanks,


(Jan Doberstein) #2

that is currently not possible - as the pipeline runs during ingest.

You would need to create a search, run that search. Possible via script and cron.


(Philipp Ruland) #3

If you are able to use plugins, this could help:

Basically build a pipeline function that does a lookup using the source field (or the field containing the host identifier, why that see below) and sorting the responses descending. The returning field should be the timestamp and then simply check if the returned timestamp is older than 3 days.

The only issue: You would still need some way of running these checks without relying on the host to send a message himself. As @jan said, you would need a script triggered by cron for example, that periodically sends messages for each host that you want to be checked. (This is were the field containing the host identifier mentioned above comes from :slight_smile:)
Also you should put this into its own index etc. to not clutter your normal logs too much :slight_smile:

Greetings,
Philipp


(system) #4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.