Recently, I had issues with my ES cluster that graylog uses. It was complaining about a lot of unassigned shards (1800 on a 6 data node cluster). Upon examination, graylog was only supposed to keep a handful of indices open but there were a lot more open than what was defined. Why isnt graylog deleting these indices once their retention/rotation policy has been met? I had to manually delete older indices to bring the cluster state back to green. I have attached screenshot to show that graylog has way more indices open and not deleted than defined (esp the daily ones).
Anybody here? Aything more I can provide?
Have you checked your log files on Graylog server/s?
How did you configure your Graylog cluster?