# Graylog stop process message, elasticsearch status ok

**URL:** <https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012>\
**Category:** Graylog Central (peer support)\
**Created:** [November 30, 2020, 3:02pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012 "2020-11-30T15:02:09Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [November 30, 2020, 3:02pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/1 "2020-11-30T15:02:09Z")

</div>

Hi, today my GL installation stop to process message. I cant find anything from the last days in the log search. I’ve try to update GL to last 3.1.\* stable version with no luck.

Elastichsearch status is ok, i can query old index message, but new message are not processed. Click on start or stop process message does nothing:

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/e/e24f46232eb1bb4326cf630d9b641dce4d222cc4.png)

---

<div class="post-metadata">

**Author:** ![aaronsachs](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/aaronsachs/32/7180_2.png) [@aaronsachs](https://community.graylog.org/u/aaronsachs)\
**Post date:** [November 30, 2020, 6:51pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/2 "2020-11-30T18:51:12Z")

</div>

@matteolavaggi your journal has hit the full watermark, so Graylog won’t process any additional messages. What’s your disk utilization look like? I’m assuming it’s full, or very near full.

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 1, 2020, 7:53am UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/3 "2020-12-01T07:53:34Z")

</div>

Hi, yesterday was full but i’ve rotate some index (es is on the same machine, i know is not a big setup but this is a temporary setup), but after rotating disk is not full:  
 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/c/c2e280232b191ea018d3303db4742708c92d64f9.png)

---

<div class="post-metadata">

**Author:** ![shoothub](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/shoothub/32/6412_2.png) [@shoothub](https://community.graylog.org/u/shoothub)\
**Post date:** [December 1, 2020, 10:05am UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/4 "2020-12-01T10:05:10Z")

</div>

Check if your cluster is not read only.

> **[Cluster Blocks Read-Only in Elasticsearch - An Important Guide](https://opster.com/elasticsearch-glossary/cluster-blocks-read-only/)**
>
> There are two types of blocks that can be applied to clusters for different reasons. Understanding what they each do and when they should be used is key.

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 1, 2020, 11:31am UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/5 "2020-12-01T11:31:10Z")

</div>

Index are ok and not locked i’ve check, recheck and unlocked again for sure ^^

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/d/d1e1c1222c499cb1ce3793c045f799d4a1373f69.png)

ubuntu@syslog01:~ curl -XPUT -H "Content-Type: application/json" http://localhost:9200/\_all/\_settings -d '{"index.blocks.read\_only\_a llow\_delete": null}' ubuntu@syslog01:~ curl -XPUT -H “Content-Type: application/json” [http://localhost:9200/\_all/\_settings](http://localhost:9200/_all/_settings) -d ‘{“index.blocks.read\_only”: false}’

---

<div class="post-metadata">

**Author:** ![ttsandrew](https://avatars.discourse-cdn.com/v4/letter/t/97f17d/32.png) [@ttsandrew](https://community.graylog.org/u/ttsandrew)\
**Post date:** [December 1, 2020, 3:20pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/6 "2020-12-01T15:20:17Z")

</div>

Hello @matteolavaggi, just to be sure, is that space in the word “allow” in “read\_only\_allow\_delete” there in the command you’re pasting into CLI? If so it’s not going to do what it should. Easy to overlook with tired eyes.

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 2, 2020, 1:07pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/7 "2020-12-02T13:07:33Z")

</div>

No space is a copy paste from terminal issue. Command is ok and ES reply with ok status.

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 2, 2020, 1:08pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/8 "2020-12-02T13:08:46Z")

</div>

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/f/f963ac5838359957cf61e8ab585361b4b3d1e154.png)

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 4, 2020, 1:13pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/9 "2020-12-04T13:13:59Z")

</div>

Any idea ? Its so strange all work great, no error message in graylog, but log entry are notprocessed

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 4, 2020, 2:34pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/10 "2020-12-04T14:34:18Z")

</div>

NO solution =\> reinstall.  
All from scratch.

### There is a node without any running inputs. (triggered 17 minutes ago)

There is a node without any running inputs. This means that you are not receiving any messages from this node at this point in time. This is most probably an indication of an error or misconfiguration. You can click [here](http://syslog01.sbg5.pc01.ovh.pineappleitaly.com:9000/system/inputs) to solve this.

Port 1514 is not listening for income log . any idea?

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 4, 2020, 2:37pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/11 "2020-12-04T14:37:49Z")

</div>

SOLVE, the graylog configuration does not store input section. Need to manual create again!

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 14, 2020, 3:30pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/12 "2020-12-14T15:30:13Z")

</div>

New installation, new server, new elastic, after 1 week same problem.  
No log can be searched, index status green.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/c/c783bacc9356ede035239b989d8a4e0feb3d3cd5.png)

---

<div class="post-metadata">

**Author:** ![shoothub](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/shoothub/32/6412_2.png) [@shoothub](https://community.graylog.org/u/shoothub)\
**Post date:** [December 14, 2020, 4:16pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/13 "2020-12-14T16:16:01Z")

</div>

Maybe your HW is not enough (ES) to store such messages, how many messages do you store every day/second? Check your CPU, RAM and disk utilization.

---

<div class="post-metadata">

**Author:** ![aaronsachs](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/aaronsachs/32/7180_2.png) [@aaronsachs](https://community.graylog.org/u/aaronsachs)\
**Post date:** [December 14, 2020, 7:59pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/14 "2020-12-14T19:59:00Z")

</div>

@matteolavaggi is there anything in the logs at all for Elasticsearch?

---

<div class="post-metadata">

**Author:** ![cawfehman](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/cawfehman/32/4132_2.png) [@cawfehman](https://community.graylog.org/u/cawfehman)\
**Post date:** [December 15, 2020, 5:10am UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/15 "2020-12-15T05:10:46Z")

</div>

You’re journal is 96% full with 67 million messages in it. the oldest is 15 minutes old… that means you are getting about 75k message per second. But you have 1GB of heap. you need more RAM… possibly more CPUs, but most likely both.

---

<div class="post-metadata">

**Author:** ![matteolavaggi](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/matteolavaggi/32/7561_2.png) [@matteolavaggi](https://community.graylog.org/u/matteolavaggi)\
**Post date:** [December 15, 2020, 2:58pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/16 "2020-12-15T14:58:56Z")

</div>

Message rate is not so high, i think this is an error on how graylog show the total message in journal and the oldest data count. Log rate is about 3-4 k every minutes

Also i found that 2/3 of total error related to log message come from ES read only option / allow delete. Why dont include a fix for this directly in graylog gui?

![](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/f/fe01dcd98fa0fb5e6b903ae330e4d1f5413fd65c.gif)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [December 29, 2020, 2:59pm UTC](https://community.graylog.org/t/graylog-stop-process-message-elasticsearch-status-ok/18012/17 "2020-12-29T14:59:07Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
