I have defined an index strategy that creates one index per day with an index retention of 365 but it doesn’t work anymore, the logs will be written on the same index, the rotation doesn’t work anymore.
I am in version 4.2.13 for Graylog and 7.16.2 for Elasticsearch. Graylog is installed on one server and Elasticsearch on another. I can’t find anything in the log files.
I already tried to recalculate the indexes but without success
Thank you for your help !
Hey @Antoine_L
From your Screen shot it look like the last Range re-calulated was a year ago. Have you tried to manually rotating you index set? There has to be something in the logs, ES/GL, Mongo, service logs.Perhaps try to restart Graylog service and TAIL -f graylogs log file?
Second, Im not sure if you know but Graylog 4.2 use Elasticsearch 7.10, many have gone past this required version, just a thought.
Thanks for your answer, I just clicked on the “rotate active write index” button, what is the command to recalculate the indexes manually?
I already restarted the graylog service but without success. Nothing in the logs of my graylog server either, I will look in the logs of my elasticsearch server.
Do you think this could be due to the too high version of elasticsearch? Can we go back to a previous version?
Hey @Antoine_L
Yes
I really dont know, it could be a couple different reasons. Looking at your total shard operation it took 17 days, that is a very long time, Im just assuming here but for 1 year( 365 days) 4 shards per day 1460 shards, that is alot of shards.
The number of shards you can hold on a node will be proportional to the amount of heap you have available, but there is no fixed limit enforced by Elasticsearch. A good rule-of-thumb is to ensure you keep the number of shards per node below 20 per GB heap it has configured. A node with a 30GB heap should therefore have a maximum of 600 shards, but the further below this limit you can keep it the better. This will generally help the cluster stay in good health.
Thank you for your answer, what exactly is a fragment? I did not understand your answer very well.
I will look in the elasticsearch logs to see if there is something
If not, is it possible to revert to a previous version without impacting the functioning of graylog?
Thanks
Hey @Antoine_L
I you refering to this here?
Ill requote my statement.
Looking at your total shard operation it took 17 days, that is a very long time.
As shown above what you posted, your retention is for 1 year( 365 days) with 4 shards per index that is 1460 shards, that is alot of shards. This mean you may be oversharding. Keep in mind that the number of shards you can hold on a node will be proportional to the amount of heap you have available, but there is no fixed limit enforced by Elasticsearch. A good rule-of-thumb is to ensure you keep the number of shards per node below 20 per GB heap it has configured. A node with a 30GB heap should therefore have a maximum of 600 shards, but the further below this limit you can keep it the better. This will generally help the cluster stay in good health.
This is a good read about that.
If you need to revert Elasticsearch and/or Opensearch to an earlier version you might with a lot of work, and it may not work at all depending on how far back you want to go.
hope that helps
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
