# Range recalculation on index rotation **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [August 23, 2019, 9:10am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687 "2019-08-23T09:10:07Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 9:10am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/1 "2019-08-23T09:10:07Z")
Hello. I’m using Graylog-3.1.0, but i had the same issue with 3.0.2. I’ve must have make a mistake somewhere, but can’t find it. I have to manually recalculate index ranges each time an index is rotated (i only have the default index set), otherwise i can’t see new messages in either streams. Disable index optimization after rotation is not checked in configuration. Thanks for any pointers. ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [August 23, 2019, 9:39am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/2 "2019-08-23T09:39:41Z")
Did you manipulate ES outside of Graylog? Is that cluster using authentication or similar (based on x-pack or searchguard)? Did you see anything in your logs that indicates that something did not work as it should? ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 10:21am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/3 "2019-08-23T10:21:33Z")
ES is used with grafana & kibana for metrics in another indices. I may have read once or twice the graylog indices with kibana and configure an index pattern for that. Otherwise, no auth, and unfortunatly nothing in the logs (and i lost the logs in graylog 3.1, nothing in event streams or in graylog.log). It does not happen when i rotate the index manually. ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [August 23, 2019, 10:37am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/4 "2019-08-23T10:37:41Z")
we would need to ability to reproduce that or identify in any way what happens. Can you check the logs after/during rotation of the indices? The complete Cycle should be something like: ```auto 2019-08-20T02:00:07.800+02:00 INFO [AbstractRotationStrategy] Deflector index (index set ) should be rotated, Pointing deflector to new index now! 2019-08-20T02:00:07.805+02:00 INFO [MongoIndexSet] Cycling from to . 2019-08-20T02:00:07.805+02:00 INFO [MongoIndexSet] Creating target index . 2019-08-20T02:00:07.825+02:00 INFO [Indices] Successfully created index template graylog-internal 2019-08-20T02:00:07.933+02:00 INFO [MongoIndexSet] Waiting for allocation of index . 2019-08-20T02:00:07.938+02:00 INFO [MongoIndexSet] Index has been successfully allocated. 2019-08-20T02:00:07.938+02:00 INFO [MongoIndexSet] Pointing index alias to new index . 2019-08-20T02:00:07.960+02:00 INFO [SystemJobManager] Submitted SystemJob <78abb270-c2dd-11e9-9f34-7a9860de6ce3> [org.graylog2.indexer.indices.jobs.SetIndexReadOnlyAndCalculateRangeJob] 2019-08-20T02:00:07.960+02:00 INFO [MongoIndexSet] Successfully pointed index alias to index . ``` The System Job is what runs the index range calculation. ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 12:24pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/5 "2019-08-23T12:24:49Z")
Logs are quite strange, I have nothing from august 4th to 21th, despite the fact my graylog\_7 index has been create 4 days ago: graylog\_7 Contains messages from 4 days ago up to 2 hours ago Hence, I have nothing for this rotation. Graylog 3.1.0 does not logs anymore in graylog-server.log. In the console log, I have the following, on august 21, when I updated graylog to 3.1, realize I did not have any log in streams and recalculate the ranges: 2019-08-21 16:29:25,891 INFO : org.graylog2.system.jobs.SystemJobManager - Submitted SystemJob \<13a21e20-c420-11e9-ba76-0cc47a306f45\> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] 2019-08-21 16:29:25,891 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Recalculating index ranges. 2019-08-21 16:29:25,898 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Recalculating index ranges for index set Default index set (graylog\_\*): 6 indices affected. 2019-08-21 16:29:27,158 INFO : org.graylog2.indexer.ranges.MongoIndexRangeService - Calculated range of [graylog\_6] in [1256ms]. 2019-08-21 16:29:27,187 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Created ranges for index graylog\_6: MongoIndexRange{id=null, indexName=graylog\_6, begin=2019-08-04T03:52:00.720Z, end=2019-08-19T19:56:04.740Z, calculatedAt=2019-08-21T14:29:25.901Z, calculationDuration=1256, streamIds=[000000000000000000000001, 5d303decd6816b304090b0c6, 5d30408fd6816b304090b3ba]} 2019-08-21 16:29:28,555 INFO : org.graylog2.indexer.ranges.MongoIndexRangeService - Calculated range of [graylog\_4] in [1365ms]. 2019-08-21 16:29:28,556 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Created ranges for index graylog\_4: MongoIndexRange{id=null, indexName=graylog\_4, begin=2019-07-11T17:15:11.600Z, end=2019-07-22T17:55:25.844Z, calculatedAt=2019-08-21T14:29:27.189Z, calculationDuration=1365, streamIds=[000000000000000000000001, 5d039e38d6816b304060586f, 5d303decd6816b304090b0c6, 5d30408fd6816b304090b3ba, 5ce40155d6816b65fe037e36]} 2019-08-21 16:29:29,910 INFO : org.graylog2.indexer.ranges.MongoIndexRangeService - Calculated range of [graylog\_3] in [1351ms]. 2019-08-21 16:29:29,911 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Created ranges for index graylog\_3: MongoIndexRange{id=null, indexName=graylog\_3, begin=2019-07-02T07:14:20.792Z, end=2019-07-11T17:15:10.605Z, calculatedAt=2019-08-21T14:29:28.558Z, calculationDuration=1351, streamIds=[000000000000000000000001, 5d039e38d6816b304060586f, 5ce40155d6816b65fe037e36]} 2019-08-21 16:29:31,394 INFO : org.graylog2.indexer.ranges.MongoIndexRangeService - Calculated range of [graylog\_5] in [1481ms]. 2019-08-21 16:29:31,395 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Created ranges for index graylog\_5: MongoIndexRange{id=null, indexName=graylog\_5, begin=2019-07-22T17:55:26.298Z, end=2019-08-04T03:51:59.725Z, calculatedAt=2019-08-21T14:29:29.913Z, calculationDuration=1481, streamIds=[000000000000000000000001, 5d303decd6816b304090b0c6, 5d30408fd6816b304090b3ba, 5d4190e5d6816b3040a3778a]} 2019-08-21 16:29:32,780 INFO : org.graylog2.indexer.ranges.MongoIndexRangeService - Calculated range of [graylog\_2] in [1383ms]. 2019-08-21 16:29:32,781 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Created ranges for index graylog\_2: MongoIndexRange{id=null, indexName=graylog\_2, begin=1973-04-04T02:50:59.000Z, end=2019-07-02T07:14:19.807Z, calculatedAt=2019-08-21T14:29:31.397Z, calculationDuration=1383, streamIds=[000000000000000000000001, 5d039e38d6816b304060586f, 5ce40155d6816b65fe037e36, 5c112171d6816b11dc00af75]} 2019-08-21 16:29:32,783 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index ranges for 6 indices. Took 6885ms. 2019-08-21 16:29:32,784 INFO : org.graylog2.system.jobs.SystemJobManager - SystemJob \<13a21e20-c420-11e9-ba76-0cc47a306f45\> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 6892ms. ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 2:07pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/6 "2019-08-23T14:07:42Z")
Could it be related to a restart of MongoDB without restarting graylog? ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [August 23, 2019, 2:35pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/7 "2019-08-23T14:35:33Z")
> 2019-08-21 16:29:32,783 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index ranges for 6 indices. Took 6885ms. - What is the load of your system? - how many shards did you have in your elasticsearch? - do you share one system for everything? ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 2:42pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/8 "2019-08-23T14:42:43Z")
1. very very low, 0.4/0.2/0.2 for a 10 cores system 2. 130 shards on a single machine 3. yes, mongo+es+graylog ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [August 23, 2019, 2:55pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/9 "2019-08-23T14:55:17Z")
what kind of storage did you use? that is really long run for index recalc on such a system. ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 23, 2019, 3:00pm UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/10 "2019-08-23T15:00:25Z")
One spinning disk, SATA 4TB on XFS. It’s a long run, but eventually succeed when i launch it manually ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [August 26, 2019, 8:31am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/11 "2019-08-26T08:31:27Z")
it might be I/O related - but I can’t say for sure. From what you have written, it looks like the duration of the recalculation takes longer than it should. ---
**Author:** ![zorel](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/zorel/32/4754_2.png) [@zorel](https://community.graylog.org/u/zorel)\ **Post date:** [August 26, 2019, 8:38am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/12 "2019-08-26T08:38:31Z")
From my point of view, it should fails when launched by hand too, but it does not. However, I understand there is no configuration for this, I can’t have made a mistake changing a parameter? ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [September 9, 2019, 8:43am UTC](https://community.graylog.org/t/range-recalculation-on-index-rotation/11687/13 "2019-09-09T08:43:18Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.