Website starts, but 404-Error for API-Call

Graylog Central (peer support)
1

The spaces in the URLs are due to the fact that the amount is limited to max. 2 URLs in a posting.

Thank you for your time, attention and help.

Graylog works generally.
I can access it via the IP-Adress on the bash-shell via cli.

Via the subdomain graylog.domain.tld it also starts to load the web interface, but than stops with a 404 error for the API call.

Error message
cannot GET htt ps: // graylog .domain.tld/api/ (404)

So I assume it must be an NGINX-Error.

I followed the Setup-Guideline for Ubuntu 20.04 LTS as documented here:
htt ps :// docs .graylog .org/en/latest/pages/installation/os/ubuntu.html

server.log states everything is fine:

2021-01-12T17:37:03.022+01:00 INFO  [ServiceManagerListener] Services are healthy
2021-01-12T17:37:03.023+01:00 INFO  [InputSetupService] Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2021-01-12T17:37:03.033+01:00 INFO  [ServerBootstrap] Graylog server up and running.
2021-01-12T17:37:03.034+01:00 ERROR [AuditLogger] Unable to write audit log entry because there is no valid license.

Calling Graylog API via CLI Calls works well too:

curl -i htt p: // 127 . 0.0.1:9000/api/

RESULT

HTTP/1.1 200 OK
X-Graylog-Node-ID: 0ac30c9f-1c97-4998-a277-86172c64f714
X-Runtime-Microseconds: 2199
Content-Type: application/json
Content-Length: 232

{"cluster_id":"5711a116-b55c-464c-9fa4-eeb4dda23c98","node_id":"0ac30c9f-1c97-4998-a277-86172c64f714","version":"4.0.1+6a0cc0b","tagline":"Manage your logs in the dark and have lasers going and make it look like you're from space!"}

Running with NGINX - Proxy via SSL as documented here
htt ps: // docs . graylog . org/en/4.0/pages/configuration/web_interface.html#webif-connecting-to-server

# Quelle htt ps: // docs . graylog . org/en/4.0/pages/configuration/web_interface.html
# for Nginx

location / {
	proxy_set_header Host $http_host;
	proxy_set_header X-Forwarded-Host $host;
	proxy_set_header X-Forwarded-Server $host;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Graylog-Server-URL https://$server_name/;
	proxy_pass http://127.0.0.1:9000;
	#	proxy_ignore_headers "Cache-Control" "Expires" "X-Accel-Expires" "Set-Cookie";
}

The proxy_error_log of nginx states:

2021/01/12 18:13:49 [error] 816844#0: *2435 "/var/www/vhosts/domain.tld/graylog.domain.tld/api/index.html" is not found (2: No such file or directory), client: 85.1232.221.95, server: domain.tld, request: "GET /api/ HTTP/2.0", host: "graylog.domain.tld", referrer: "https://domain.tld/"

Eventhough I don’t believe it’s an configuration issue here is the server.conf

############################
# GRAYLOG CONFIGURATION FILE
############################
#
is_master = true

node_id_file = /etc/graylog/server/node-id

# The minimum length for "password_secret" is 16 characters.
password_secret = ....


# Default: bin
bin_dir = /usr/share/graylog-server/bin

data_dir = /var/lib/graylog-server

# Set plugin directory here (relative or absolute)
plugin_dir = /usr/share/graylog-server/plugin

# Default: 127.0.0.1:9000
http_bind_address = 127.0.0.1:9000

#### HTTP publish URI
# Default: http://$http_bind_address/
#http_publish_uri = http://graylog.domain.tld:9000/

#### External Graylog URI

# Default: $http_publish_uri
#http_external_uri = https://graylog.domain.tld/

#TEST-Weise, wohl eher veraltete Informationen...
#rest_listen_uri = https://graylog.domain.tld/api
#web_listen_uri = https://graylog.domain.tld/

#### Enable CORS headers for HTTP interface
#
# This allows browsers to make Cross-Origin requests from any origin.
# This is disabled for security reasons and typically only needed if running graylog
# with a separate server for frontend development.
#
# Default: false
#http_enable_cors = false

#### Enable GZIP support for HTTP interface
http_enable_gzip = true
2

Hi there,

It’s a bit strange that you’re getting all those 404’s. What does your full nginx config look like?

3

Hello,

thank you for your question and willingness to help to analyse this issue.
We do use Plesk for Administration of the server, so it’s a chain of different files for the configuration, I have glued the differenz files to one large one.

By the way:
Direct access via HTTP://123.321.222.111:9000 works fine.

So this points to an nginx-configuration issue.

Source:

EDITED AND ERASED FOR BETTER READABILITY
4

Found the error: A missing slash behind the port.

You have that correct in the Apache-Example, but it’s missing in the nginx-part of your documentation:
https://docs.graylog.org/en/4.0/pages/configuration/web_interface.html

location / {
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header Host $http_host;
	proxy_set_header X-Forwarded-Host $host;
	proxy_set_header X-Forwarded-Server $host;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Graylog-Server-URL https://$server_name/;
	proxy_pass http://123.321.222.111:9000/;
}
5

Hmmmm, I’m a bit skeptical that the slash at the end was the problem. Take for example this working nginx config that I use in my lab:

upstream graylog {
  server logs00.example.com:9000 max_fails=3 fail_timeout=30s;
  server logs01.example.com:9000 max_fails=3 fail_timeout=30s;
  server logs02.example.com:9000 max_fails=3 fail_timeout=30s;
}

server {
  listen *:80;
  server_name           logs.example.com;

  return 301            https://$host$request_uri;
  access_log            /var/log/nginx/logs.example.com.access.log combined;
  error_log             /var/log/nginx/logs.example.com.error.log;
}

server {
  listen       *:443 ssl;
  server_name  logs.example.com;

  ssl_certificate           /etc/nginx/ssl/fullchain.pem;
  ssl_certificate_key       /etc/nginx/ssl/privkey.pem;
  ssl_session_cache         shared:SSL:10m;
  ssl_session_timeout       5m;
  ssl_protocols             TLSv1.2;
  ssl_ciphers               ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS;

  ssl_prefer_server_ciphers on;

  index  index.html index.htm index.php;

  access_log /var/log/nginx/ssl-logs.example.com.access.log combined;
  error_log  /var/log/nginx/ssl-logs.example.com.error.log;

  location / {
    # Simple requests
    if ($request_method ~* "(GET|POST)") {
      add_header "Access-Control-Allow-Origin"  *;
    }

    # Preflighted requests
    if ($request_method = OPTIONS ) {
      add_header "Access-Control-Allow-Origin"  *;
      add_header "Access-Control-Allow-Methods" "GET, POST, OPTIONS, HEAD";
      add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
      return 200;
    }

    proxy_pass http://graylog;
    proxy_redirect http://graylog:443/api /api;
    proxy_read_timeout 90;
    proxy_connect_timeout 90;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Graylog-Server-URL https://$server_name/;
  }
}

The proxy_pass directive doesn’t use a slash. I suspect this is is because in your config there’s no upstream, you’re just passing directly to the IP address.

I’ll also note that Plesk seems to be adding quite a lot of extraneous bits in the config, unless that’s just from you stitching the configs together. Here’s what I’m referencing:

server {
	listen 123.123.123.123:443 ssl http2;

	server_name graylog.example.org;
	server_name www.graylog.example.org;
	server_name ipv4.graylog.example.org;

	ssl_certificate             /opt/psa/var/certificates/example_cert.pem;
	ssl_certificate_key         /opt/psa/var/certificates/example_key.pem;

	error_page 400 "/error_docs/bad_request.html";
	error_page 401 "/error_docs/unauthorized.html";
	error_page 403 "/error_docs/forbidden.html";
	error_page 404 "/error_docs/not_found.html";
	error_page 500 "/error_docs/internal_server_error.html";
	error_page 405 "/error_docs/method_not_allowed.html";
	error_page 406 "/error_docs/not_acceptable.html";
	error_page 407 "/error_docs/proxy_authentication_required.html";
	error_page 412 "/error_docs/precondition_failed.html";
	error_page 414 "/error_docs/request_uri_too_long.html";
	error_page 415 "/error_docs/unsupported_media_type.html";
	error_page 501 "/error_docs/not_implemented.html";
	error_page 502 "/error_docs/bad_gateway.html";
	error_page 503 "/error_docs/maintenance.html";

	location ^~ /error_docs {
		root "/var/www/vhosts/example.org";
	}

	client_max_body_size 128m;

	root "/var/www/vhosts/example.org/graylog.example.org";
	access_log "/var/www/vhosts/system/graylog.example.org/logs/proxy_access_ssl_log";
	error_log "/var/www/vhosts/system/graylog.example.org/logs/proxy_error_log";

	#extension letsencrypt begin
	location ^~ /.well-known/acme-challenge/ {
		root /var/www/vhosts/default/htdocs;

		types { }
		default_type text/plain;

		satisfy any;
		auth_basic off;
		allow all;

		location ~ ^/\.well-known/acme-challenge.*/\. {
			deny all;
		}
	}
	#extension letsencrypt end

	#extension sslit begin

	#extension sslit end

	location ~ /\.ht {
		deny all;
	}

	location ~ ^/(plesk-stat|awstats-icon|webstat|webstat-ssl|ftpstat|anon_ftpstat) {
		auth_basic "Domain statistics";
		auth_basic_user_file "/var/www/vhosts/system/graylog.example.org/pd/d..httpdocs@plesk-stat";
		autoindex on;

		location ~ ^/plesk-stat(.*) {
			alias /var/www/vhosts/system/graylog.example.org/statistics/$1;
		}

		location ~ ^/awstats-icon(.*) {
			alias /usr/share/awstats/icon/$1;
		}

		location ~ ^/(.*) {
			alias /var/www/vhosts/system/graylog.example.org/statistics/$1;
		}
	}

	location ~ ^/~(.+?)(/.*?\.php)(/.*)?$ {
		alias /var/www/vhosts/example.org/web_users/$1/$2;
		fastcgi_split_path_info ^((?U).+\.php)(/?.+)$;
		fastcgi_param PATH_INFO $fastcgi_path_info;
		fastcgi_pass "unix:///var/www/vhosts/system/graylog.example.org/php-fpm.sock";
		include /etc/nginx/fastcgi.conf;

	}

	location ~ \.php(/.*)?$ {
		fastcgi_split_path_info ^((?U).+\.php)(/?.+)$;
		fastcgi_param PATH_INFO $fastcgi_path_info;
		fastcgi_pass "unix:///var/www/vhosts/system/graylog.example.org/php-fpm.sock";
		include /etc/nginx/fastcgi.conf;

	}

	location ~ /$ {
		index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";
	}

	add_header X-Powered-By PleskLin;

	include "/var/www/vhosts/system/graylog.example.org/conf/vhost_nginx.conf";
##
##FROM FILE: /var/www/vhosts/system/graylog.example.org/conf/vhost_nginx.conf
##
#
# Quelle https://docs.graylog.org/en/4.0/pages/configuration/web_interface.html
# for Nginx

      location / {
 	proxy_set_header Host $http_host;
	proxy_set_header X-Forwarded-Host $host;
	proxy_set_header X-Forwarded-Server $host;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Graylog-Server-URL https://$server_name/;
	proxy_pass http://123.123.123.123:9000;
      }

}

^ That config is a mess. It references things that Graylog doesn’t even use (e.g., php, the error docs, etc). I get that there’s likely a genuine need for Plesk, but in this case, it certainly doesn’t seem to be helping, or making things any clearer.

6

Thank you for your time and attention and support.

The behaviour with and without the slash at the end leads to a reproducable behaviour.

The PHP can be disabled, it doen’t have impact anyway as only the proxy functionality of nginx is used and relevant.

So this maybe helpful for others who use plesk for the general administration too.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.