1. Describe your incident:
I am looking for a working configuration to run the official graylog-datanode docker image when /proc/sys/vm/max_map_count is set to 65530 and cannot be changed.
2. Describe your environment:
-
OS Information: Railway (cloud container platform)
-
Package Version: 6.3.1 Docker images
-
Service logs, configurations, and environment variables:
Contents of /etc/graylog/datanode/overrideconfig.conf:
node.store.allow_mmap=false
Contents of /etc/graylog/datanode/datanode.conf:
opensearch_configuration_overrides_file = /etc/graylog/datanode/overrideconfig.conf
Datanode startup log:
2025-08-01T19:18:47.655Z INFO [cluster] Waiting for server to become available for operation with ID 3. Remaining time: 29977 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=mongodb-7c80a5be.railway.internal:27017, type=UNKNOWN, state=CONNECTING}].
2025-08-01T19:18:47.677Z INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=mongodb-7c80a5be.railway.internal:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=40973186, minRoundTripTimeNanos=0}
2025-08-01T19:18:47.846Z INFO [DatanodeDirectories] Opensearch of the node e5325ad9-9355-40ef-b558-802c140a795c uses following directories as its storage: DatanodeDirectories{dataTargetDir='/var/lib/graylog-datanode/opensearch/data', logsTargetDir='/var/lib/graylog-datanode/opensearch/logs', configurationSourceDir='Optional.empty', configurationTargetDir='/var/lib/graylog-datanode/opensearch/config'}
2025-08-01T19:18:47.927Z INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "5.5.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.12.12+bpo-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.15+6"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='mongo', source='graylog', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7650ded6, com.mongodb.Jep395RecordCodecProvider@5034f5dd, com.mongodb.KotlinCodecProvider@31b7d869]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[mongodb-7c80a5be.railway.internal:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
2025-08-01T19:18:47.931Z INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver", "version": "5.5.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.12.12+bpo-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.15+6"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='mongo', source='graylog', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7650ded6, com.mongodb.Jep395RecordCodecProvider@5034f5dd, com.mongodb.KotlinCodecProvider@31b7d869]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[mongodb-7c80a5be.railway.internal:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
2025-08-01T19:18:47.937Z INFO [cluster] Waiting for server to become available for operation with ID 11. Remaining time: 29999 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=mongodb-7c80a5be.railway.internal:27017, type=UNKNOWN, state=CONNECTING}].
2025-08-01T19:18:47.940Z INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=mongodb-7c80a5be.railway.internal:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=9719612, minRoundTripTimeNanos=0}
2025-08-01T19:18:48.032Z INFO [MongoDBPreflightCheck] Connected to MongoDB version 7.0.20
2025-08-01T19:18:48.090Z ERROR [PreflightCheckService] Preflight check failed with error: /proc/sys/vm/max_map_count value should be at least 262144 but is 65530 (set via "vm.max_map_count" sysctl)
2025-08-01T19:18:48.091Z ERROR [CmdLineTool] Startup error:
org.graylog2.bootstrap.preflight.PreflightCheckException: /proc/sys/vm/max_map_count value should be at least 262144 but is 65530 (set via "vm.max_map_count" sysctl)
at org.graylog.datanode.bootstrap.preflight.OpenSearchPreconditionsCheck.runCheck(OpenSearchPreconditionsCheck.java:51) ~[graylog-datanode.jar:?]
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:421) ~[guava-33.4.8-jre.jar:?]
at org.graylog2.bootstrap.preflight.PreflightCheckService.runChecks(PreflightCheckService.java:52) ~[graylog2-server-6.3.1.jar:?]
at org.graylog.datanode.bootstrap.DatanodeBootstrap.runPreFlightChecks(DatanodeBootstrap.java:82) ~[graylog-datanode.jar:?]
at org.graylog.datanode.bootstrap.DatanodeBootstrap.beforeInjectorCreation(DatanodeBootstrap.java:70) ~[graylog-datanode.jar:?]
at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:362) ~[graylog2-server-6.3.1.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:287) [graylog2-server-6.3.1.jar:?]
at org.graylog.datanode.bootstrap.Main.main(Main.java:59) [graylog-datanode.jar:?]
Exception in thread "main" org.graylog2.bootstrap.preflight.PreflightCheckException: /proc/sys/vm/max_map_count value should be at least 262144 but is 65530 (set via "vm.max_map_count" sysctl)
at org.graylog.datanode.bootstrap.preflight.OpenSearchPreconditionsCheck.runCheck(OpenSearchPreconditionsCheck.java:51)
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:421)
at org.graylog2.bootstrap.preflight.PreflightCheckService.runChecks(PreflightCheckService.java:52)
at org.graylog.datanode.bootstrap.DatanodeBootstrap.runPreFlightChecks(DatanodeBootstrap.java:82)
at org.graylog.datanode.bootstrap.DatanodeBootstrap.beforeInjectorCreation(DatanodeBootstrap.java:70)
at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:362)
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:287)
at org.graylog.datanode.bootstrap.Main.main(Main.java:59)
3. What steps have you already taken to try and solve the problem?
I reported a datanode bug here which has been fixed (thanks Tdvorak.) Based on the release tags it seems the fix is in 6.3.1 but I am still having problems.
4. How can the community help?
By helping me find a working configuration for the official Docker image.