Proof of Concept: Index Retention "Do Nothing"

[accidentaladmin (Matthew)]

Good Afternoon:

I am testing Graylog 6.0 on my homelab before putting it into production at work. I would like to implement a hot/cold architecture for my indices. I know this isn’t possible if sticking with Graylog Open’s GUI. So, I have installed Opensearch Dashboard.

In OS Dashboard, I have created an Index Management Policy where all new indices are allocated to “Hot” nodes, whereas upon reaching 35 days, transferred to “Cold” Nodes and closed.

So as not to cause a “fight” between OS and Graylog, I have set Index retention within Graylog to “Do Nothing”. Well, first I set it to “Index Time Optimizing” and then set retention to “Do Nothing”.

For existing indices that are older than 35 days, the new policy has worked like a charm. That being said, I haven’t witnessed the creation of new indices within Graylog and am beginning to get nervous as to what may happen.

1. Will Graylog create any new Indices or will it just keep writing to the existing index until OS closes it.
2. If it will just continue to write until closed by OS, will it then create a new index?

Predictions / Informed Opinions are welcome.

Thank you!

[patrickmann]

Based on your rotation strategy, GL should rotate after a max of 35 days (or earlier, if index is getting too big). If index is moved to cold before GL has had a chance to rotate, that might be a problem. How about transferring to cold at 36 days?