Allow graylog in docker to use multiple cores


I would like to ask if there is a way to allow java to spawn its threads to multiple cores when I run a dockerized graylog image.

Expected Behavior

I would like to use a specific number of cores that are available in my systems.

Current Behavior

Right now java uses only 2 cores while there are 10 in each node. Docker can identify that there are 10 cores in the system.

Possible Solution

Any JAVA_OPTS flag that could enable that feature

Steps to Reproduce (for bugs)


There is a huge amount of messages arriving in the graylog cluster and the cluster is unable to process them. I have tried to adjust the [input,output,process]buffer_processors but this didn’t help. I think that the worst thing is that there are available resources but java doesn’t use them. This results to a very huge load (like 20-22), 100% usage at the first 2 cores of the system and also to the filling up of the journal.

Graylog is also installed in a different cluster without using docker and I’m not encountering this issue. The java threads are using all the cores.

Your Environment

  • processbuffer_processors = 5
  • outputbuffer_processors = 3
  • inputbuffer_processors = 2
  • message_journal_max_size = 20gb
  • GRAYLOG_SERVER_JAVA_OPTS=-Xms1g -Xmx4g -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow
  • Graylog Version: 2.4.3-3
  • Elasticsearch Version: 5.6.7
  • MongoDB Version: 3.4.13
  • Operating System: Centos-release-7
  • Browser version:

Which version of Docker are you using?
Which Docker image are you using?
What’s the complete configuration of the Docker container?
What’s the complete configuration of Graylog?

I just replaced the passwords and some urls.

Try using graylog/graylog:2.4.5-1.

The JVM used in the old graylog2/server Docker image doesn’t support Docker (see You’ll also have to update your GRAYLOG_SERVER_JAVA_OPTS environment variable accordingly.

Also see:

I have tried using the same JAVA_OPTS and using much simpler ones as well -Xms1g -Xmx4g -XX:+UnlockExperimentalVMOptions but it didnt work. Only 2 CPUs are used.

Both is wrong. Please use the default value for GRAYLOG_SERVER_JAVA_OPTS as template for your customization.

Are you sure that the Docker container is allowed to use more CPUs?
See Runtime options with Memory, CPUs, and GPUs | Docker Documentation for details.

What’s the full output of the following command?

# docker inspect $CONTAINER_ID | grep -i cpu


$ docker inspect 654142f112ae | grep -i cpu
            "CpuShares": 0,
            "NanoCpus": 0,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "CpuCount": 0,
            "CpuPercent": 0,

Even the default + -Xms1g -Xmx4g isn’t working.

# docker inspect 0a81b1e53578 | grep -i cpu
            "CpuShares": 0,
            "NanoCpus": 0,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "CpuCount": 0,
            "CpuPercent": 0,

That’s not the default for the new Docker image and as long as you don’t provide the complete and unredacted value of GRAYLOG_SERVER_JAVA_OPTS you’re using, we won’t be able to help you.

What’s the complete output of the following command?

# docker system info

So I’ve tried using
-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow


-Xms1g -Xmx4g -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow

but both are not working.

Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 3
Server Version: 18.03.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
  Profile: default
Kernel Version: 3.10.0-862.2.3.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 10
Total Memory: 7.693GiB
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Experimental: false
Insecure Registries:
Live Restore Enabled: false

Please run the Docker image groovy:2.5.0-jre8 and post the output when executing the expression Runtime.runtime.availableProcessors().


$ docker run -it groovy:2.5.0-jre8
Jun 06, 2018 3:46:46 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Groovy Shell (2.5.0, JVM: 1.8.0_171)
Type ':help' or ':h' for help.
groovy:000> Runtime.runtime.availableProcessors()
===> 4
groovy:000> :exit

I get this response. I guess something is wrong with the docker engine (?)

# docker run -it groovy:2.5.0-jre8
Unable to find image 'groovy:2.5.0-jre8' locally
2.5.0-jre8: Pulling from library/groovy
cc1a78bfd46b: Already exists
d2c05365ee2a: Pull complete
231cb0e216d3: Pull complete
e8912f9d0ce2: Pull complete
9bafe362f99b: Pull complete
28b0652112a6: Pull complete
da23e1e20eae: Pull complete
9d809d99b239: Pull complete
7ff0806e176a: Pull complete
b0ab2872bc72: Pull complete
Digest: sha256:130410986a5ef8db430f7f89c1397c41d11347a8ecc47a38e89155896c35d35b
Status: Downloaded newer image for groovy:2.5.0-jre8
Jun 07, 2018 6:48:04 AM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Groovy Shell (2.5.0, JVM: 1.8.0_171)
Type ':help' or ':h' for help.
groovy:000> Runtime.runtime.availableProcessors()
===> 2

Try explicitly assigning the number of CPU cores you want to use in the container and try again:

Thanks for all the help. I have found the issue. It had to do with systemd and not with java or dockerd. It was my mistake. I changed the resources of the VMs but didn’t restart dockerd to refresh the cgroup limits.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.