I am seeing this weird behavior where the index set rotates before it reaches the size that is set. For e.g. I had the index set size set to 1 gb before rotating to the next index set but it started getting rotated at around 400mb. I tried changing the size of index set to 530mb but it still rotates short of that. I will attach screenshots below.
I have checked the elasticseach cluster and status is green.
In case any of you has seen this before any help will be much appreciated!
2. Describe your environment:
OS Information: Red Hat Enterprise Linux Server release 7.9 (Maipo)
Package Version: 4.3.15+17ed3ac, codename Noir
Service logs, configurations, and environment variables:n/a
I’m curious what indexer are you using? (Elasticsearch/OpenSearch and version)
We no longer recommend using size based rotation, but i suspect this could possibly be due to index compression. More details about your environment are needed though.
It also looks like you are running an old version of Graylog from a few years ago. We’ve just released Graylog 6.1. If you get a chance you should check it out
You have Index optimization enabled on this index set. This is a compression process that runs after an index has been rotated, which in my experience typically compresses an index to around 60% of its original size (but note compression rate varies depending on the data within the index).
If your index is rotating at 1GB, it seems within ballpark that the compressed data is ~400MB.
If you disable index optimization, you should find your rotated indices are closer to the 1GB size.
