I have a question about long term log retention. I’m using and All-In-One Graylog server that has 500GB drive. Graylog is configured for index rotation of 1 Day, Delete index, 90 Max number of indices. I needed to trend some data from last year but unfortunately, I can only go back 90 days.
I’m fully aware of Archiving functionality in Graylog Enterprise. I’m also aware of elasticsearch Index lifecycle management (ILM) policies to automatically manage, but I need to accomplish this in that way. What I was looking at is keeping my index rotation strategy as is, but instead of deleting indices after 90 days I would the close indices out. Maybe I could move the closed indices to a long-term storage (JBOD/NAS). I’m not sure if this works and unsure if I can index them again after moving them. I have also looked at Elasticsearch Snapshot, not sure if it would work in my instance. Looking for some suggestion that I can use that’s not costly. If anyone has option or suggestion that I could use it would be much appreciated.
Thank in advance.