Hello,
My apologies, I misread your post.
- I believe Its because of this statement in your logs.
low disk watermark [85%] exceeded on
- Definition of low disk watermark [85%] exceeded.
When disk usage on a host hits 85 percent, the Elasticsearch service prevents shard allocation and stops working. This disk usage threshold is an Elasticsearch configuration. By default, the cluster.routing.allocation.disk.watermark.low watermark is set to 85% to prevent Elasticsearch from allocating new shards to hosts once disk usage on the host exceeds 85 percent.
- The Next stage is High Disk Watermark
The next level is the High Disk Watermark stage. The default value is 95%. Elasticsearch enforces a read-only index block (index.blocks.read_only_allow_delete) on every index that has one or more shards allocated on the node, and that has at least one disk exceeding the flood stage. This setting is the last resort to prevent nodes from running out of disk space. Depending on your Elasticsearch version, the release mechanism is different:
-
Before Elasticsearch 7, the index block must be released manually when the disk utilization falls below the high watermark.
-
Since Elasticsearch 7, the index block is automatically released when the disk utilization falls below the high watermark.
-
Maybe these post will enlighten you on what’s going on.
Hope that helps