Hello guys, i need your help. Im struggling with an incident on my prod env. I have a HPE ProLiant ML350 Gen9 64GB RAM 16CPU. I have 2 VMs installed in Proxmox, Graylog 4 in SSD and Windows Server 2016 DC. Graylog VM specs: [1] I have 78 configured inputs in Graylog, windows (GELF TCP), linux (s…

Hello, AUDIT SUCCESS Event ID 4663 ( An attempt was made to access an object) I have this in my lab , for security measures. [image] To be honest if this is messing things up for you I would look into the source of this, instead of trying to patch it with pipelines, just an idea. [image] T…

It could be an errant GROK on a message that has changed slightly - here is a relevant post to that: [image] Graylog nodes stop outputting/fill up buffers Graylog Central (peer support) The original data was full of PII, I compiled a minimal example. All this was part of…

[image] tmacgbay: It could be an errant GROK on a message that has changed slightly - here is a relevant post to that: As i have noticed some times this happens in the morning when all employees come to work and turn on their computers. The indicator in the top left corner [22] indicates 30…

I have a MUCH smaller environment and company and for me it was a GROK pattern that wasn’t matching the string properly and got lost… could have been backtracking like mentioned in the other post… though it was a simple change for me on a field match of IP and/or IPORHOST. GROK is helped greatly if…

I also looked the proxmox graphs for any peaks on freeze time but nothing abnormal. The graphs are still, no cpu or memory pekas to indicate something. I will check all the grogs again for any change.

When my system was locked up, I don’t recall it topping out memory or CPU. I don’t know for sure at all if it is your issue, just one possibility of many.

Take a look at graylog and jvm config for any missconfiguration i have missed. I will post the nxlog config tomorrow when i get to work

Hello, Just chiming in. I noticed you stated this when employees come to work. [image] T1000-Cyberd: indicates 3000+ in and 3000+ out Windows has been known to create message storms. If your GL freezes up during that time, have you checked your buffers when this issue occurs? If they are a…

Nope no spikes of memory or cpu. Also the buffers are not filling up at that moment. I has happened at nights too so not only when working hours start. How its possible to freeze the entire host, and not only the GL VM. Maybe can i change something in the configs to test it out? Any idea?

OK i think i might have a clue. I audit file systems on my windows machines, but now i remember having a big storm of messages “An attempt was made to access an object.” AUDIT SUCCESS when the antivirus scan kicked in. The AV scan starts on most of the devices at the same time, so GL is flooded wi…

Hello, and thanks for the reply. This is the solution, reducing and filtering the messaes from the source. Thanks again.

Graylog 4.0.15 Freezes proxmox host randomly

Graylog Central (peer support)

tmacgbay (Tmacgbay) February 24, 2022, 2:53pm 2

It could be an errant GROK on a message that has changed slightly - here is a relevant post to that:

Topic		Replies	Views
Graylog resets entire host VM Graylog Central (peer support)	1	56	March 1, 2025
Graylog nodes stop outputting/fill up buffers Graylog Central (peer support)	14	6394	April 22, 2020
Slow down the Graylog system and not all inputs are process Graylog Central (peer support)	41	6609	April 6, 2017
Graylog server extremely slow. cant get System/inputs Graylog Central (peer support)	18	6651	November 20, 2017
Struggling with Graylog stopping to export to Elasticsearch Graylog Central (peer support) pipeline-rules , debuggingpl	13	2143	July 22, 2021

Graylog 4.0.15 Freezes proxmox host randomly

Related topics