Hi all,
I have a Input from our CGNAT and need to store data retention for 1 year for IP usage.
How can I set this, and what is the most efficient configuration for Index Rotation Configuration and Index Retention Configuration.
Im not sure what value should be used.
Thank you!
How much data are you going to be receiving every day, both the number of messages and the total size of those messages?
It would be 40,000 messages per day, but Im not sure how to see the size of each one. Where can I find that info? Sorry im new to graylog.
If you are already ingesting the messages you can just post a picture of the graph from the system>overview page, the will show volume for the last 30 days.
As its only recent I can only get the daily which is 3800M bytes
The image below is for 6 customers at the moment. 3800M would be for around 200 customers per day.
That seems managable for one year 
- put your messages all in a unique stream
- create an index set with the weekly rotation (P1W), 53 Max number of indices, rotation strategy based on Index Time
- configure your stream to use the indexset
Make sure to maybe split your opensearch-database above two machines, and to configure Index replicas to 1. One of your machines might die, and you will not lose data.
If your amount of data is growing it will be more of a challenge, but ~100MB/day should be fine.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.