OptimizeIndexJob times out

#1

hi,

I have the following errors:

2017-03-02T02:01:03.458+02:00 INFO  [SystemJobManager] SystemJob <336ccc80-fedb-11e6-88d8-0050568617f3> [org.graylog2.indexer.indices.jobs.SetIndexReadOnlyAndCalculateRangeJob] finished in 23913ms.
2017-03-02T03:00:40.395+02:00 ERROR [SystemJobManager] Unhandled error while running SystemJob <45d022a0-fedb-11e6-88d8-0050568617f3> [org.graylog2.indexer.indices.jobs.OptimizeIndexJob] org.elasticsearch.ElasticsearchTimeoutException: Timeout waiting for task.

Wherein lies the problem? Should I increase the number of Elasticsearch servers, set up some Elasticsearch parameter, some Graylog parameter or something else?

(Jochen) #2

The index optimization process in Elasticsearch took longer than the configured (default) time out for this operation, which is 1 hour.

See elasticsearch_index_optimization_timeout.

This is usually a problem with slow I/O or huge indices. Either way, providing faster storage (e. g. using SSDs) can tremendously speed up the process.

If you know what you’re doing, you can also increase the time out for the optimization process or disable index optimization altogether.

#3

Thank you!

The indices are about 150G currently, but I expect that to grow to somewhere near 1T per index, when everything is running properly.

Currently I run with creating a daily index - I wonder if changing the retention properties to something like PT3H would actually solve the problem?

(Jochen) #4

It might help. There’s no better way to find out than to try it. :wink:

#5

… I’ll try :slight_smile:

Thanks.