Hey all,
What is the suggested practice for storing / indexing long term metadata?
I have several hourly, daily, weekly and even monthly dashboards - but in some cases, like Firewall traffic - I don’t want to store a million messages per day for a year. I only really need the full message for a few days, however I’d like to keep an eye on trends and report on KPIs quarterly or annually.
I’d rather summarize week by week and have the ability to simply produce the quantities.
Is there a way to store the count of a search query somehow, then index that message for a year? Is anyone else doing this? Things like DNS, webserver logs, firewall logs produce huge quantities - and I only really need the quantity after a certain period of time.