# Indexing of new messages stoppes occasionally **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [May 5, 2017, 3:42pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058 "2017-05-05T15:42:46Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 5, 2017, 3:42pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/1 "2017-05-05T15:42:47Z")
I have a Graylog Cluster with 2 GL Servers and 3 Elasticsearch Servers. I have the Problem that sometimes Graylog just stops indexing new messages into Elasticsearch although ES is online. To fix it I usually have to restart the Elasticsearch Processes. Sometimes it works fine for a week or two and sometimes it only works for 3 days. Reproducing this doesn’t seem possible. Where should I look at to see what the source of this is? ---
**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\ **Post date:** [May 5, 2017, 3:49pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/2 "2017-05-05T15:49:21Z")
Any warnings or errors in the logs of your Graylog and Elasticsearch nodes (see [http://docs.graylog.org/en/2.2/pages/configuration/file\_location.html](http://docs.graylog.org/en/2.2/pages/configuration/file_location.html))? ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 10, 2017, 6:58pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/3 "2017-05-10T18:58:19Z")
Gotta wait until the error happens again… It’s not actualy reproducable so it may take a while. ---
**Author:** ![rafaelcarsetimo](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/rafaelcarsetimo/32/215_2.png) [@rafaelcarsetimo](https://community.graylog.org/u/rafaelcarsetimo)\ **Post date:** [May 10, 2017, 7:03pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/4 "2017-05-10T19:03:09Z")
The stuck occurs in both GL Servers at the same time? ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 11, 2017, 4:33pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/5 "2017-05-11T16:33:39Z")
Also; do your servers have enough memory? ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 12, 2017, 12:52am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/6 "2017-05-12T00:52:19Z")
Yes both servers are stopping the processing. ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 12, 2017, 12:54am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/7 "2017-05-12T00:54:25Z")
The Graylog Servers have 4vCPU/8GB the Elasticsearch(3) Servers have 8vCPU/16GB each ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 12, 2017, 7:18am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/8 "2017-05-12T07:18:29Z")
and how much did you allocate to graylog and elasticsearch JVM:s? My guess would be for Elasticsearch max 8G, for Graylog max 2G or 3G ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 14, 2017, 10:21pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/9 "2017-05-14T22:21:32Z")
Graylog 4GB each ES 8GB each ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 14, 2017, 10:39pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/10 "2017-05-14T22:39:16Z")
Just had that error again. For whatever reason there were no Logfiles for Elasticsearch for a few days. But there are files like these on all 3 ES servers. I had to kill the ES Process to restart them. -rw------- 1 elasticsearch elasticsearch 5171027968 May 14 18:24 java\_pid23573.hprof ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 15, 2017, 5:01am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/11 "2017-05-15T05:01:50Z")
Try this [http://www.eclipse.org/mat/](http://www.eclipse.org/mat/) ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 15, 2017, 5:05am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/12 "2017-05-15T05:05:22Z")
Btw. Are you using the recommended Java version (Oracle Java 1.8) in the Elasticsearch nodes? ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 15, 2017, 12:54pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/13 "2017-05-15T12:54:51Z")
○ → java -version java version "1.8.0\_45" Java™ SE Runtime Environment (build 1.8.0\_45-b14) Java HotSpot™ 64-Bit Server VM (build 25.45-b02, mixed mode) ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 15, 2017, 7:07pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/14 "2017-05-15T19:07:56Z")
45 sounds pretty old. The current version is 131. I don’t know if a relevant bug has been fixed in between, but at least it should not make things worse. To me this sounds like your Elasticsearch crashes and dumps, so it does not seem to be a graylog problem. Btw - I guess you use Elasticsearch 2.4 series version (compatible with Graylog) ? Have you installed any Elasticsearch plugins - removing them might also help. ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 16, 2017, 12:06am UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/15 "2017-05-16T00:06:52Z")
only elasticsarch-head. I will try to update the JRE and see if it helps ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 16, 2017, 5:44pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/16 "2017-05-16T17:44:36Z")
Problem occured again. There are several messages like the following. These occur on all 3 ES Servers New used memory 5368893176 [5gb] for data of [source] would be larger than configured breaker: 5112122572 [4.7gb], breaking ---
**Author:** ![jtkarvo](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jtkarvo](https://community.graylog.org/u/jtkarvo)\ **Post date:** [May 17, 2017, 6:03pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/17 "2017-05-17T18:03:32Z")
> [@claudekenni](#): > > New used memory 5368893176 [5gb] for data of [source] would be larger than configured breaker: 5112122572 [4.7gb], breaking you run out of memory in the Elasticsearch nodes. Look at this article: [https://www.elastic.co/guide/en/elasticsearch/guide/current/\_limiting\_memory\_usage.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html) ---
**Author:** ![claudekenni](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/claudekenni/32/331_2.png) [@claudekenni](https://community.graylog.org/u/claudekenni)\ **Post date:** [May 17, 2017, 7:33pm UTC](https://community.graylog.org/t/indexing-of-new-messages-stoppes-occasionally/1058/18 "2017-05-17T19:33:35Z")
got it. Will see if tweaking these setting will help with the issue. Thanks for the info