# Problem elasticsearch **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** pipeline-rules\ **Created:** [November 17, 2020, 5:17pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873 "2020-11-17T17:17:44Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 17, 2020, 5:17pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/1 "2020-11-17T17:17:45Z")
Hello everyone, I collect a very large amount of logs, when I do a search (search in last 5, 10.15 minutes …) I do not receive the logs. (just for information, my server logs are powerful, i.e. it is not a resource problem on the server side) **While retrieving data for this widget, the following error(s) occurred:** - Connection refused (Connection refused). Please Help !!! ![problem elasticsearch critical](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/1/1112bbbc8798297197adf18f1573abba9e74d917.png) In this case, is it necessary Add more resources to Elasticsearch or adjust the output settings from Graylog to Elasticsearch ?? Can somebody please explain it to me ? ---
**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\ **Post date:** [November 17, 2020, 5:44pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/2 "2020-11-17T17:44:56Z")
Hello @Labidi, Based on the messages it looks like either your elasticsearch instance/cluster is at the least not accessible, but also possibly not healthy or not able to keep up (resources) with the amount of data it is receiving. It appears the graylog server is journaling messages more quickly than elasticsearch can process them, hence the warning about some messages being flushed from the journal before they can be indexed. Make sure ES is reachable by Graylog. If it is, check health and performance. Good luck! ---
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 17, 2020, 6:50pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/3 "2020-11-17T18:50:07Z")
Hello @ttsandrew Thanks for your feedback, ES is reachable by Graylog and is **green** ![ayoub status EA](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/1/1c75594c79b2779ae81161bb41d4681067e3787b.png) I need a solution. the problem that I receive a lot of logs ---
**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\ **Post date:** [November 17, 2020, 6:56pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/4 "2020-11-17T18:56:42Z")
Is ES served up on the same host as graylog? Or is it a separate host? If they are on separate hosts then the messages queueing on Graylog more quickly than they can be indexed by ES could be a problem with the network connection between them. How is performance of Graylog? Do you have any pipeline rules: ---
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 17, 2020, 7:02pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/5 "2020-11-17T19:02:56Z")
Graylog and elasticsearch in the same host ! I assure you that there is not a network problem, graylog has enough ram 12 GB and 8 vcpu and has large amount of storage… ---
**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\ **Post date:** [November 17, 2020, 7:04pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/6 "2020-11-17T19:04:23Z")
Ok. So where is the issue? Do you have any pipeline rules? Is the storage able to keep up with both journaling and indexing messages? ---
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 17, 2020, 7:06pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/7 "2020-11-17T19:06:59Z")
now it is working properly. its operation is correct I can load (at least 5 minutes …) and is ok, NO I don’t have a pipeline rules 😕 . of course, I indexed the logs for a period of 4 months, I indexed it per month, but it’s getting heavy. so I indexed it daily, (for a period of 4 months) ---
**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\ **Post date:** [November 17, 2020, 7:30pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/8 "2020-11-17T19:30:01Z")
If there is no evidence of compute exhaustion, there’s no network issue to worry about, and you don’t have any special processing like a pipeline rule then I would suspect storage performance. You mention that you are processing a lot of messages. If you look at iotop/atop/top (wa column) do you see many processes waiting for disk? Related, we found this guide very valuable for monitoring performance to identify any issues: > **[Monitoring Graylog - Host Metrics that you should Monitor Regularly | Graylog](https://www.graylog.org/post/back-to-basics-monitoring-graylog)** We also followed the Elasticsearch guidelines for sizing based on our deployment to ensure that we are within or as near to as possible the recommendations for shard count / size and adjusted Graylog indices as necessary. ---
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 17, 2020, 7:47pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/9 "2020-11-17T19:47:54Z")
I’ll check the document and get back to you , thanks @ttsandrew ---
**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\ **Post date:** [November 18, 2020, 9:59pm UTC](https://community.graylog.org/t/problem-elasticsearch/17873/10 "2020-11-18T21:59:11Z")
For posterity, this was resolved over in this thread. > [@Graylog blocked](https://community.graylog.org/t/graylog-blocked/17795/): > > Hello , Suddenly my graylog is crashed, I cannot receive the palo alto firewall logs or no others Vm linux for my infrastructure - Please help ---
**Author:** ![Labidi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/labidi/32/5597_2.png) [@Labidi](https://community.graylog.org/u/Labidi)\ **Post date:** [November 19, 2020, 7:49am UTC](https://community.graylog.org/t/problem-elasticsearch/17873/11 "2020-11-19T07:49:52Z")
yes , thanks a lot @ttsandrew ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [December 3, 2020, 7:49am UTC](https://community.graylog.org/t/problem-elasticsearch/17873/12 "2020-12-03T07:49:53Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.