# Problems in graylog about Disk Journal and Process buffer **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [February 16, 2021, 12:10pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816 "2021-02-16T12:10:14Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 16, 2021, 12:10pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/1 "2021-02-16T12:10:14Z")
hi everyone, I have the version of graylog 3.2.6 and I have the following errors: - I only have 1 node - Process buffer → 65536 messages in process buffer, 100.00% utilized. - Output buffer → 65536 messages in output buffer, 100.00% utilized. - Disk Journal → 101.51% 3,704,904 unprocessed messages are currently in the journal, in 53 segments. 0 messages have been appended in the last second, 0 messages have been read in the last second. - Memory/Heap usage → The JVM is using 816.8MiB of 972.8MiB heap space and will not attempt to use more than 972.8M to finish I have configured a stream and an index-set with: Shards: 4 Replicas: 0 Field type refresh interval: 5 seconds Index rotation strategy: Index Size Max index size: 1073741824 bytes (1.0GiB) Index retention strategy: Delete Max number of indices:10 and graylog : - cpu consumption by graylog java is: 86 % - the machine had 4 cpu and 8 memory and jvm -\>4g - outputbuffer\_processors = 3 - processbuffer\_processors = 5 more dates: “status” : “green”, “timed\_out” : false, “number\_of\_nodes” : 1, “number\_of\_data\_nodes” : 1, “active\_primary\_shards” : 84, “active\_shards” : 84, “relocating\_shards” : 0, “initializing\_shards” : 0, “unassigned\_shards” : 0, “delayed\_unassigned\_shards” : 0, “number\_of\_pending\_tasks” : 0, “number\_of\_in\_flight\_fetch” : 0, “task\_max\_waiting\_in\_queue\_millis” : 0, “active\_shards\_percent\_as\_number” : 100.0 more dates: “size\_in\_bytes” : 6708393602 memory\_size\_in\_bytes" : 9405192, mem" : { “total\_in\_bytes” : 8345530368, “free\_in\_bytes” : 3799244800, “used\_in\_bytes” : 4546285568, “free\_percent” : 46, “used\_percent” : 54 “process” : { “cpu” : { “percent” : 0 }, “open\_file\_descriptors” : { “min” : 2040, “max” : 2040, “avg” : 2040 “mem” : { “heap\_used\_in\_bytes” : 336385816, “heap\_max\_in\_bytes” : 3186360320 }, “threads” : 48 Many many thanks, ---
**Author:** ![aaronsachs](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/aaronsachs/32/7180_2.png) [@aaronsachs](https://community.graylog.org/u/aaronsachs)\ **Post date:** [February 16, 2021, 12:41pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/2 "2021-02-16T12:41:08Z")
…your disk journal is full. What’s disk utilization look like on that box? If your disk is full, then Graylog’s gonna have a bad time. ---
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 16, 2021, 12:56pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/3 "2021-02-16T12:56:08Z")
> [@aaronsachs](#): > > What’s disk utilization look like on that box? hi, sorry me, how can I give you the information you ask for?. What I can tell you is that I don’t have the parameters configured in graylog. Would you have to configure them? #message\_journal\_max\_age = 12h #message\_journal\_max\_size = 5gb #message\_journal\_flush\_age = 1m #message\_journal\_flush\_interval = 1000000 #message\_journal\_segment\_age = 1h #message\_journal\_segment\_size = 100mb thanks, [image] ---
**Author:** ![aaronsachs](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/aaronsachs/32/7180_2.png) [@aaronsachs](https://community.graylog.org/u/aaronsachs)\ **Post date:** [February 16, 2021, 1:51pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/4 "2021-02-16T13:51:35Z")
Hey there. Run `df -h` on that system. What does it say? We’re not talking about Graylog itself at this point. It’s your system’s disk. ---
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 16, 2021, 2:02pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/5 "2021-02-16T14:02:01Z")
hi , there are no space problems. the fs are 44% free. regards. ---
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 17, 2021, 9:23am UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/6 "2021-02-17T09:23:13Z")
hi everyone, I have already managed to lower the journal disk, increasing the space. But now, the big question is: How can I clean the buffer? or How can I modify the buffer so that it is not 100%? , - Output buffer → 65536 messages in output buffer, 100.00% utilized. → solution ? - Process buffer → 65536 messages in process buffer, 100.00% utilized. → solution ? output\_batch\_size 500 processbuffer\_processors = 4 outputbuffer\_processors = 7 input → Receive Buffer Size → 1048576 , → should i increase it? Can you please help me ? Many many thanks much, ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [February 17, 2021, 11:04am UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/7 "2021-02-17T11:04:29Z")
he @elpedrop you should check the elasticsearch log. I guess you have the worker queue full as your Graylog is connecting with up to 7 connections at the same time … Lower the `outputbuffer_processor` to 3 and raise the `output_batch_size` to 1500 this should allow your Graylog to handover the messages to elasticsearch in time. In addition you should create a custom mapping for Elasticsearch that sets the `index_refresh` rate to ~30 seconds. That will give you a performance boost at all. ---
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 17, 2021, 12:01pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/8 "2021-02-17T12:01:31Z")
hi jan , How can I make the change in elastic of index, how is it done? dynamically? or or in the elastisearch.yml file? many thanks ---
**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\ **Post date:** [February 17, 2021, 12:04pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/9 "2021-02-17T12:04:59Z")
he @elpedrop you have multiple options for that. Once is via the custom mapping as described here: [https://docs.graylog.org/en/4.0/pages/configuration/elasticsearch.html#custom-index-mappings](https://docs.graylog.org/en/4.0/pages/configuration/elasticsearch.html#custom-index-mappings) but your favorite search engine will give you more options. ---
**Author:** ![elpedrop](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/elpedrop/32/7951_2.png) [@elpedrop](https://community.graylog.org/u/elpedrop)\ **Post date:** [February 17, 2021, 3:55pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/10 "2021-02-17T15:55:21Z")
hi , sorry, it already seems to work. Many thanks, ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [March 3, 2021, 3:55pm UTC](https://community.graylog.org/t/problems-in-graylog-about-disk-journal-and-process-buffer/18816/11 "2021-03-03T15:55:54Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.