Suggestions for Long Term trend analysis

Hey all,

What is the suggested practice for storing / indexing long term metadata?

I have several hourly, daily, weekly and even monthly dashboards - but in some cases, like Firewall traffic - I don’t want to store a million messages per day for a year. I only really need the full message for a few days, however I’d like to keep an eye on trends and report on KPIs quarterly or annually.

I’d rather summarize week by week and have the ability to simply produce the quantities.

Is there a way to store the count of a search query somehow, then index that message for a year? Is anyone else doing this? Things like DNS, webserver logs, firewall logs produce huge quantities - and I only really need the quantity after a certain period of time.

He ya,

not yet - but upcoming Graylog version will have that. That you can correlate and save the correlated data as a new message. Stay tuned for the next release!

Currently you would need to create the denser messages on processing saving it in a different index and keep that longer.

1 Like

Cool. I’ll watch for the new features. Thanks as always, @jan

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.