Graylog 3.1 Appliance, High CPU 100%

from few days our Graylog Appliance (in the morning) runs with 100% CPU (4vCPU) for some hours then returns in normal state. There aren’t log rotation operations (this happend every 7d, in the weekend) and there aren’t a lot of INCOMING logs (no more then 200 msg/s). Currently there are 7 index with 3 shards. RAM 12 GB. Buffer is full (100%, 65536 msg), Journaling is growing (current is 2%). How I can check wich operation is “blocking” the node?

Which buffer? Graylog has 3 buffers, and it can tell, what part has a problem.

Process buffer is 100%, Utilization Journal is growing (slowly)

Do you use Grok Patterns? Maybe it is a runaway grok pattern. In Graylog nodes stop outputting/fill up buffers you might find a solution.

image I don’t have the entry for Get Process-buffer Dump.

I have resolved stopping few minutes the UDP 514 INPUT, then after the restart all seem to be processed well… question: How I can find which message bloccked the processing?

with the given version no way - that is the reason later versions have the ability to make a processing buffer dump where you see the worker threads and messages they are working on.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.