Hi,
I’ve got strange behavior related to indices rotation.
One of our instances has an index set with size-based rotation, like 100 indexes 500Mb each.
Active write index (say, graylog_98) is 0 size.
Previous index (say, graylog_97) is almost 100Gb and keep growing.
All previous indexes (96, 95…1) are 500Mb as expected.
So for me it looks like as a kind of indexer issue.
Manual deletion of over-sized index via Graylog REST API solved the problem and rotation works fine now.
Unfortunately I have no case how to reproduce it. It happened just one single time.
It is production setup, so I wonder what can be a root cause to prevent similar incidents in the future.
Does anybody faced something similar?