Http 400 after upgrade to Graylog 5.2.3

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
I upgraded a working Graylog Enterprise installation today to the latest version 5.2.3. I am able to login into Graylog, but after that I get http 400 responses for all subsequent request. A valid license is installed.
I do not see any error message in the logfiles with a hint for the reason.

2. Describe your environment:

  • OS Information:
    Ubuntu 20.04.6

  • Package Version:
    I had a Graylog 5.1.6 installation using elastic search 7.2 and MongoDB 5.0.24 and tried to upgrade to Graylog 5.2.3. I updated using:

wget https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.deb
sudo dpkg -i graylog-5.2-repository_latest.deb
sudo apt-get update
sudo apt-get install graylog-enterprise

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?
Reboots, Restart, Scanning the logfiles

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

What version of the JVM do you have, and what do you get if you check the status of the Graylog service?

3

Thanks for your response.

I am using the included JVM, that ist “Eclipse Adoptium 17.0.9” or

openjdk version “17.0.9” 2023-10-17
OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode, sharing)

reported on the command line.

sudo systemctl status graylog-server.service

returns

graylog-server.service - Graylog server
Loaded: loaded (/lib/systemd/system/graylog-server.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2024-01-27 18:45:28 CET; 16h ago
Docs: http://docs.graylog.org/
Main PID: 63405 (graylog-server)
Tasks: 167 (limit: 38067)
Memory: 1.3G

and

sudo systemctl status elasticsearch.service

Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2024-01-27 18:59:10 CET; 17h ago
Docs: https://www.elastic.co
Main PID: 67371 (java)
Tasks: 84 (limit: 38067)
Memory: 2.7G
CGroup: /system.slice/elasticsearch.service
└─67371 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -D>

sudo systemctl status mongod.service
mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2024-01-27 15:07:20 CET; 20h ago
Docs: https://docs.mongodb.org/manual
Main PID: 535 (mongod)
Memory: 270.5M
CGroup: /system.slice/mongod.service
└─535 /usr/bin/mongod --config /etc/mongod.conf

And I see 2 other messages in /var/log/graylog-server/server.log:

WARN [LookupTableService] Unable to load data adapter watchlist-mongo of type mongodb, missing a factory. Is a required plugin missing?
WARN [LookupTableService] Lookup table watchlist is referencing a missing data adapter 61ae04c47c4cd047cfc17221, check if it started properly.

But I have a second installation, same environment, with the same messages and that server ist running fine.

4

Out of sheer desperation and because I had run out of ideas, I set the maximum header size in Graylog from 8192 to 16384. And now it’s working again. A log entry would have been really helpful.

5

Do you mean http_max_header_size in server.conf?

6

Yes. Setting this to a higher value solved my problem.

Graylog is running behind an ALB in AWS and an ID-Provider. That results in a lot of headers. However, I do not know why it was working with Graylog 5.1.x and not with 5.2.3

7

That’s very interesting, it’s something we haven’t seen as of yet so it you could open a GitHub issue it would be helpful for the engineering team to look into.GitHub - Graylog2/graylog2-server: Free and open log management

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.