Extreme hardware usage

lucappelman · November 8, 2024, 3:34pm

Last week I installed graylog on an virtual machine for testing, everything went great and the performance was good. I followed the docker installation steps only changing the value of some (private) keys.

Today I did the same on a new virtual machine for production to a similarly capable machine. Unfortunately I get really high read disk usage after just a couple minutes. This essentially causes the entire VM to lock up and having to reset the VM from the dashboard of my VPS host.
I installed iotop and tried again, it seemed like the MongoDB container was the cause so I updated the container version to the latest 6.x.x version by changing the tag. After this the issue still persists.

I haven’t even been able to finish configuring Graylog past some inputs and streams. I haven’t configured any of my services yet to send logs to Graylog so there’s incoming data for Graylog to process.

Is my current installation broken? Is the hardware not powerful enough?

Hardware:
2 CPU cores
4GB RAM
150 GB SSD

OS: Ubuntu 24 LTS
Docker version: 27.3.1, build ce12230
Graylog version: 6.1

lucappelman · November 8, 2024, 3:57pm

Perhaps some extra logs would be useful. This is up untill the server becomes completely unresponsive.

couldn’t upload *.txt so I created a github gist

gist.github.com

https://gist.github.com/controlol/2d9b15113f813b563df19a6601a08cfe

graylog-install-problem

 ✔ Container graylog-mongodb-1   Created                                                                                                                                                                      0.0s
 ✔ Container graylog-datanode-1  Created                                                                                                                                                                      0.0s
 ✔ Container graylog-graylog-1   Created                                                                                                                                                                      0.0s
Attaching to datanode-1, graylog-1, mongodb-1
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.350+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.359+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.361+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.363+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.364+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
mongodb-1   | {"t":{"$date":"2024-11-08T15:35:02.364+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}

This file has been truncated. show original

Wine_Merchant · November 11, 2024, 9:42am

Hey @lucappelman,

If you are running Graylog/Mongo and Opensearch on a single host then the resources you listed would likely be insufficient. If you were to double the CPU count and RAM, does the issue still occur?

system · November 25, 2024, 9:43am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
System Performance - Docker Single Host Increasing Interrupts Graylog Central (peer support)	10	879	December 21, 2020
Graylog container pegging 2 cpu cores for hours Graylog Central (peer support)	15	118	July 9, 2024
Graylog memory usage is high New to Graylog Community? READ-ME FIRST Guides	1	26	November 15, 2024
CPU max on single Node Cluster Graylog Central (peer support)	3	664	September 24, 2023
VM Appliance Performance Issues Graylog Central (peer support)	2	799	August 22, 2017

Extreme hardware usage

Related topics