What I would like to acheive is the send an alert if it crosses certain percentage instead of count.
For example lets say I have 1000 inbound messages in the last 10 minutes. All messages will contain either of these two strings in the message field - “Login Failed” OR “Login Success”. Lets say 100 messages contain the string “Login Failed” and the rest 900 messages have the string “Login success”.
Thats 10% “Login Failed” of total requests.
If someone can guide me how I can configure an alert if the threshold crosses 10%? I am ok if you can guide some solutions using pipelines, different streams etc. I am not able to figure out how to approach such problem statement.
I’m not sure you want to use percentages for alerting on login failures versus a fixed number of failed login attempts over a period of time.
Going with your above example, if there are 1000 login attempts, 100 failed is surely significant, but what if there’s only 10 login attempts? Is 1 significant? What if of the 100 failed login attempts, it’s really 50 people that forgot their passwords and each only had 2 login attempts? Is that significant? What if of the 1000 login attempts, there are only 80 failed attempts, but all 80 are against a single account? What if it’s a weekend and there’s only 1 person in the office and they mistype their password. Then the second attempt they login successfully. That’s 50% failure…
If you set a threshold that isn’t a percentage and then setup an alert to say ANY accounts that have more than 5 login failures in a 2 minute SPAN should fire off an alert, then you have something that’s actionable… you have a targeted alert that has times, usernames and number of attempts. And if the alert keeps firing… then it arguably warrants more attention. In the above example, you would get an alert that might not be as actionable because you will need to further parse the alert to determine who in the alert body needs addressing. Perhaps multiple people… perhaps no one.
As I don’t really know your use case, this is just some food for thought.
thank you for your reply. I should have been more precise about my specific case. I’m dealing with large quantities of events that show a some deviations over time but are correlated to each other (little risk for dividing by zero und thus breaking math). A change in their ratio gives some indication that something is going the wrong way.
So I’m looking for a way to do some math with the events and use it for triggering an alarm.
