Graylog no longer works following an upgrade to Graylog Server 7.1.4

[reni]

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:

After upgrading from Graylog Server 7.0.x to 7.1.4, it no longer works. However, the upgrade went ahead without any error messages.

2. Describe your environment:

• OS Information: Ubuntu 22.04.5 LTS

• Package Version: graylog-server/stable,now 7.1.4-1 amd64
  mongodb-org/jammy/mongodb-org/7.0,now 7.0.37 amd64
  opensearch/stable,now 2.19.5 amd64

• Service logs, configurations, and environment variables:
  These messages appear when the status is:

  systemctl status graylog-server
  ● graylog-server.service - Graylog server
  Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled)
  Active: active (running) since Thu 2026-07-02 08:17:59 CEST; 13min ago
  Docs: http://docs.graylog.org/
  Main PID: 6817 (graylog-server)
  Tasks: 104 (limit: 37575)
  Memory: 563.3M
  CPU: 42.022s
  CGroup: /system.slice/graylog-server.service
  ├─6817 /bin/sh /usr/share/graylog-server/bin/graylog-server
  └─6818 /usr/share/graylog-server/jvm/bin/java -Xms1g -Xmx1g -server -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -Djdk.tls.acknowledgeCloseNotify=true -Dlog4j2.formatMsgNoLookups=true -jar -Dlog4j.configurationFile=file:///etc/graylog/server/log>
  
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:589)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:190)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:113)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at com.google.inject.Guice.createInjector(Guice.java:87)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.shared.bindings.GuiceInjectorHolder.createInjector(GuiceInjectorHolder.java:34)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.bootstrap.CmdLineTool.doCreateInjector(CmdLineTool.java:597)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.bootstrap.CmdLineTool.setupInjector(CmdLineTool.java:592)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:364)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:287)
  Jul 02 08:18:10 lv-as-log103 graylog-server[6818]:         at org.graylog2.bootstrap.Main.main(Main.java:57)
  

3. What steps have you already taken to try and solve the problem?

No changes after a reboot, and Apache2 is also running without any problems. Nevertheless, when I try to access the website, I get a “Service Unavailable” message.

4. How can the community help?

What could be the problem following the upgrade?

Best regards

[reni]

/usr/share/graylog-server/jvm/bin/java --version
openjdk 21.0.11 2026-04-21 LTS
OpenJDK Runtime Environment Temurin-21.0.11+10 (build 21.0.11+10-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.11+10 (build 21.0.11+10-LTS, mixed mode, sharing)

The Java version should be fine and is included with Graylog Server anyway.
But what should the permissions and ownership of the files be?
I’ve read here and there that you need to adjust the ‘Owner:Group’ settings.

[Tdvorak (Tomas Dvorak)]

Could you please look at the server log and post it here, at least the full exception, which is visible partially in your systemctl status graylog-server output?

[reni]

Where exactly can I find the log?

[Tdvorak (Tomas Dvorak)]

Usually, it’s this file: /var/log/graylog-server/server.log

[reni]

20260702_server.pdf (685.7 KB)

I had to turn it into a PDF.
It’s only from 2 July 26. I did the update today; everything was working fine before that.

[reni]

20260702_server.pdf (518.9 KB)

I’ve just renamed the log file to .pdf.
Once you’ve downloaded it, you can rename it back to .log, and it’ll probably be easier to read than the converted PDF.

[Tdvorak (Tomas Dvorak)]

You seem to be affected by this problem: 7.1 regression: server fails to start on noexec data dir (oshi/JNA UnsatisfiedLinkError) · Issue #26471 · Graylog2/graylog2-server · GitHub

The recommended workaround is for now:

Point Graylog’s native-library directory at an exec-capable location. Create one owned by the Graylog user:
mkdir -p /opt/graylog/libnative/jna
chown -R graylog:graylog /opt/graylog/libnative

Add these to the GRAYLOG_SERVER_JAVA_OPTS line in /etc/default/graylog-server (Debian/Ubuntu) or /etc/sysconfig/graylog-server (RHEL/RPM):
-Djna.tmpdir=/opt/graylog/libnative/jna -Dio.netty.native.workdir=/opt/graylog/libnative

Then restart:
systemctl restart graylog-server

Confirm your chosen path is not itself on a noexec mount
findmnt -T /opt/graylog

[reni]

Yes

Here is my addition:
#Fix nach Update-Problem - 20260702
GRAYLOG_SERVER_JAVA_OPTS="$GRAYLOG_SERVER_JAVA_OPTS -Djna.tmpdir=/opt/graylog/libnative/jna -Dio.netty.native.workdir=/opt/graylog/libnative"

That helped, and Graylog is now working again.
Thank you very much!