HTTPD Running as a LB for Graylog Pair gives a white page

Graylog Central (peer support)
1

1. Describe your incident:

I’ve done an installation of two servers for Graylog, one master, one not. I have an httpd load balancer set up in front of them. When I hit the load balancer, I get a white page in response. When I hit the servers directly, I get the login page. I know I’m missing something simple, but I can’t see what, and I’m hoping I’ve understood the documentation correctly.

2. Describe your environment:

  • OS Information: Centos 7

  • Package Version: Graylog v4.2.11+ec7c16b (installed via yum)

  • Service logs, configurations, and environment variables:

Graylog server one (master) named gcp-us-east4-a-pr-graylog-01:

#graylog settings
node_id_file = /graylog/node-id
root_timezone = EST5EDT
bin_dir = /usr/share/graylog-server/bin
data_dir = /graylog/
plugin_dir = /usr/share/graylog-server/plugin
http_publish_uri = http://gcp-us-east4-a-pr-graylog-01.my.tld/
http_bind_address = 0.0.0.0:9000
http_external_uri = http://gcp-us-east4-a-pr-graylog-01.my.tld/
#IP of the load balancer
trusted_proxies = 10.128.208.126/32 
http_enable_cors = true
is_master = true

Server two (non-master) named gcp-us-east4-a-pr-graylog-02:

#graylog settings
node_id_file = /graylog/node-id
root_timezone = EST5EDT
bin_dir = /usr/share/graylog-server/bin
data_dir = /graylog/
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 0.0.0.0:9000
http_publish_uri = http://gcp-us-east4-a-pr-graylog-02.my.tld/
http_external_uri = http://gcp-us-east4-a-pr-graylog-02.my.tld/
#IP of the load balancer
trusted_proxies = 10.128.208.126/32 
http_enable_cors = true

httpd server addon config for virtual host:

<VirtualHost *:80>
    ServerName cocologger.my.tld
    ProxyRequests Off
    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    <Location />
        RequestHeader set X-Graylog-Server-URL "cocologger.my.tld"
        ProxyPass http://cocologger-graylog-01.my.tld:9000/
        ProxyPassReverse http://cocologger-graylog-01.my.tld:9000/
    </Location>

</VirtualHost>

The rest of the config is the default httpd.conf that comes with a yum install httpd (so if that’s part of the issue, I’m not well versed in httpd)

3. What steps have you already taken to try and solve the problem?

Read the docs, tested various settings in the server.conf files

4. How can the community help?

See what I might be missing?

Also, right now I’m only using 01 as the target, what do I need to do to add 02 as well? I tried having multiple ProxyPass and ProxyPassReverse lines, but it only uses the last set.

When I’ve searched the forum, most posts indicate that it’s something missing or incorrectly configured in the server.conf file, but I’m not sure what at this point, so, any help or direction would be appreciated.

I’ve confirmed the connectivity between the hosts:

[root@gcp-us-east4-a-pr-lb-httpd-01 httpd]# curl --head cocologger-graylog-01.my.tld:9000
HTTP/1.1 200 OK
X-UA-Compatible: IE=edge
X-Graylog-Node-ID: 33091279-b0ab-4b3e-b6b1-b51b3873c5f3
Content-Length: 1676
Content-Type: text/html

[root@gcp-us-east4-a-pr-lb-httpd-01 httpd]# 


[root@gcp-us-east4-a-pr-lb-httpd-01 httpd]# curl --head cocologger-graylog-02.my.tld:9000
HTTP/1.1 200 OK
X-UA-Compatible: IE=edge
X-Graylog-Node-ID: 382a3b78-0b23-40c5-9a90-6a372dfe1aa0
Content-Length: 1676
Content-Type: text/html

[root@gcp-us-east4-a-pr-lb-httpd-01 httpd]#

And on the reverse:

[root@gcp-us-east4-a-pr-graylog-01 graylog-server]# curl --head cocologger.my.tld
HTTP/1.1 200 OK
Date: Tue, 02 Aug 2022 21:33:57 GMT
Server: Apache/2.4.6 (CentOS)
X-UA-Compatible: IE=edge
X-Graylog-Node-ID: 33091279-b0ab-4b3e-b6b1-b51b3873c5f3
Content-Length: 2060
Content-Type: text/html; charset=UTF-8

[root@gcp-us-east4-a-pr-graylog-01 graylog-server]# 

[root@gcp-us-east4-a-pr-graylog-02 graylog-server]# curl --head cocologger.my.tld
HTTP/1.1 200 OK
Date: Tue, 02 Aug 2022 21:34:44 GMT
Server: Apache/2.4.6 (CentOS)
X-UA-Compatible: IE=edge
X-Graylog-Node-ID: 33091279-b0ab-4b3e-b6b1-b51b3873c5f3
Content-Length: 2060
Content-Type: text/html; charset=UTF-8

[root@gcp-us-east4-a-pr-graylog-02 graylog-server]#
2

Hello @msweikata && welcome

Only thing I can see different is the RequestHeader set X-Graylog-Server-URL

<VirtualHost *:80>
    ServerName graylog.example.org
    ProxyRequests Off
    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    <Location />
        RequestHeader set X-Graylog-Server-URL "http://graylog.example.org/"
        ProxyPass http://127.0.0.1:9000/
        ProxyPassReverse http://127.0.0.1:9000/
    </Location>

</VirtualHost>

I don’t use Apache/httpd any more but I have old configurations saved that might help.

#<VirtualHost *:80>
#    ServerName graylog.domain.com
#    ProxyRequests Off
#    <Proxy *>
#        Order deny,allow
#        Allow from all
#    </Proxy>
#    <Location />
#        RequestHeader set X-Graylog-Server-URL "http://graylog.domain.com/"
#        ProxyPass http://127.0.0.1:9000/
#        ProxyPassReverse http://127.0.0.1:9000/
#    </Location>
#</VirtualHost>




<VirtualHost *:443>
    ServerName graylog.enseva-labs.net
    ProxyRequests Off
    SSLEngine on
    SSLOptions +StrictRequire
    SSLCertificateFile "/etc/graylog/graylog3-certificate.pem"
    SSLCertificateKeyFile "/etc/graylog/graylog3-key.pem"
    RewriteEngine on
    RewriteCond %{HTTP_HOST} ^10\.10\.10\.70$ [NC,OR]
    RewriteRule .* https://graylog.domain.com:9000 [R,L]
    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    <Location />
        RequestHeader set X-Graylog-Server-URL "https://graylog.domain.com/"
        ProxyPass  https://graylog.domain.com:9000/
        ProxyPassReverse  https://graylog.domain.com:9000/
    </Location>

</VirtualHost>

Maybe check out this post

3

Hey @gsmith!

Thanks for the support.

And yes, you were 100% right, I knew it was going to be something stupid simple. Just missing that http://. Changed that, restarted httpd, boom! So that’s resolved.

So for my second question, what do I need to add to httpd to have it use both hosts in the load balanced situation? (I’m totally fine with an active/passive setup if that’s what it ends up).

I’ve tried multiple location blocks, and multiple ProxyPass lines in the same location block, but each attempt will just run to the second in the list.

4

Awesome, yeah sometime it is, I’ve done it many times.

Ummmmmmmmm :laughing: I forgot, this was like 4 years ago I used apache/httpd. We use a Fortinet firewall so I create virtual IP Address and add those to create my load balancer something like this here

Did a quick search maybe something like this here Or someone else might jump in.

5

You know, I saw that, but was in so much of a frustration about trying to get the httpd working, I must’ve just glanced over it. I’ll do a deep dive over it again, it looks like it’s just very clear on “Hey, do this step by step to make the thing do the thing.”

In your experience is there a recommended pattern of active/active, or active/standby? My gut says active/active, but I’ve not worked with Graylog in that method (usually it was just a direct connection to the host.

6

Hello,

Active-passive configuration, the server load balancer will recognizes a failed node and redirects traffic to the next available node. In an active-active configuration, the load balancer spreads out the workload’s traffic among multiple nodes so it really depends on what you want to do.
For example, with my MariaDB Galera Cluster I would use active/active, maybe not so much with Graylog because I need one master node at a time but again this would depends on the size or layout of this environment. A load balancer normally is used for a fail -over. My network has Active/active but I have a primary network port and a little bit or Byte of traffic goes out on the secondary port , this helps an uninterrupted network connection.

active/standby, if a issue occurs you may experience an interruption in the force :smiley:

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.