Confused on why I'm not storing as much as I expected

Hi there

I have graylog configured as follows

rotation_strategy = size
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_size_per_index = 10737418240
elasticsearch_max_number_of_indices = 2000
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 2

From this I work out that I should be using about 10Gx2000 = 20TB (I know, x1.5 to take into account ES indexing/etc) before the deletion of old indexes kicks in, and on top of that the replicas = 2 should mean this is using 40TB (or is it 60TB?) of diskspace aggregated on the cluster members…

But on this 5-node ES cluster, I’m only seeing 5x1.4TB of diskspace in use. It’s ‘steady state’ - I’m seeing the deletion rotation policy triggering in the logs - so obviously I’ve screwed something up. Any ideas?


Which version of Graylog are you using?

Starting with Graylog 2.2.0, the configuration of index settings has been moved into index sets (and thus into the database):

I’m on the latest via yum under CentOS-7. ie


But thanks - you’ve answered my question. I was unaware of this change. I used to have elasticsearch_max_number_of_indices set to 800 - but increased it in the config - but obviously that doesn’t work any more. But looking in the ‘System->Indices’ area, I see it’s still set to 800. Now I know what to do :slight_smile:



This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.