Apache HTTPS not working

Hi all,

Setup is a single host. Version are:

Host - Ubuntu 18.04
Graylog - 3.0.0-12
Java - 1.8.0_191
Elasticsearch - 6.6.1"
MongoDB - 4.0.6
Apache - 2.4.29

So I’m trying to set up HTTPS for my Graylog server. I have followed the guide for [Setting up the proxy](docs.http://172.24.228.161:9000cd .org/en/3.0/pages/configuration/web_interface.html#configuring-webif) and read through Ussing HTTPS

I’ve editied /etc/apache2/sites-available/default-ssl.conf and put in:

<VirtualHost *:443>

    ServerName host.domain.co.uk
    ProxyRequests Off
    SSLEngine on
    SSLCertificateFile /etc/apache2/ssl/wildtnewc.cer
    SSLCertificateKeyFile /etc/apache2/ssl/wildnew.key
    SSLCertificateChainFile /etc/apache2/ssl/wildtnewc.cer

<Proxy *>
    Order deny,allow
    Allow from all
</Proxy>

<Location />
    RequestHeader set X-Graylog-Server-URL "host.domain.co.uk:9000/"
    ProxyPass  https://host.domain.co.uk:9000/
    ProxyPassReverse  https://host.domain.co.uk:9000/
</Location> 

    </VirtualHost>

Restart apache but when I go to my host.domain.co.uk I get a error page stating

This site can’t be reached
host.domain.co.uk refused to connect.
Try:

ERR_CONNECTION_REFUSED

I can reach the site with just http://host.domain.co.uk:9000 and with IP.

There are no errors in the log file of apache, just:

mal operations
[Thu Feb 21 14:13:36.126407 2019] [core:notice] [pid 2921:tid 140311197047744] AH00094: Command line: ‘/usr/sbin/apache2’
[Thu Feb 21 14:34:33.531602 2019] [mpm_event:notice] [pid 2921:tid 140311197047744] AH00491: caught SIGTERM, shutting down
[Thu Feb 21 14:34:33.760410 2019] [mpm_event:notice] [pid 3067:tid 140199715331008] AH00489: Apache/2.4.29 (Ubuntu) configured – resuming normal operations

and there are no errors in the Graylog log, just INFO’s

Did I miss something in the config?

Hi,

Your ProxyPass directive needs to be changed from https to http. Also try with http://127.0.0.1.

Eventually recheck firewall settings.
Try “telnet” to port 443 as well.

Let me know if this helps.

B.

1 Like

Graylog can handle HTTPS itself just fine; technically speaking you don’t need to put a proxy in front of it.

Putting that aside, @bjonoski 's point looks like that would be the issue. The docs indicate the same:

http://docs.graylog.org/en/3.0/pages/configuration/web_interface.html#apache-httpd-2-x

Netcat (or nc) my dude :slight_smile: It’s the way of the future. Leave telnet behind and embrace our savior netcat :slight_smile:

Hi @bjonoski,

I have made those changed, but still not working. I have disabled the firewall for the moment.
I can get to hostname.com:9000
I can get to http://hostname.com (this takes be to the apache2 default splash screen) I can not get to https://hosntame.com ( I get ‘This site can’t be reached’)

@Totally_Not_A_Robot Is using a proxy not best practice?

I don’t know whether it’s the best practice, but I do know it’s a very common practice. Personally I believe in keeping things as simple as possible, and thought putting a proxy in-between would only muddy the waters. But that’s my personal choice :slight_smile:

So let’s get back to Netcat :slight_smile:

  • Can you connect to port 443 on your proxy? This won’t get you actual HTTPS, but it tests for the port.
  • What does openssl s_client -connect target:443 show you? Is the connection valid?
  • In the server logs for your proxy server, what does it tell you there? You should have both access and error logs for the proxy/vhost that you defined.

Thanks Tess. I went with the proxy because I’ve used apache2 in the past for Nextcloud, mediawiki and set up the SSL with no problems.

I have never set up a proxy before but from the GrayLog instructions is seemed quite straight forward. Would it be better to do this through the graylog server?

No can’t connect to 443 - Used nc -v IP 443 and get back: nc: connect to IP port 443 (tcp) failed: Connection refused
Port 80 is good. : Connection to IP 80 port [tcp/http] succeeded!
Firewall is turned off

Running openssl s_client -connect target:443 I get back:

140625505100224:error:20087002:BIO routines:BIO_lookup:system lib:…/crypto/bio/b_addr.c:693:Temporary failure in name resolution
connect:errno=1

tail /var/log/apache2/error.log is only showing notices:

[Mon Feb 25 09:30:46.116949 2019] [mpm_event:notice] [pid 14383:tid 139816477367232] AH00491: caught SIGTERM, shutting down
[Mon Feb 25 09:30:46.238871 2019] [mpm_event:notice] [pid 14522:tid 140027025042368] AH00489: Apache/2.4.29 (Ubuntu) configured – resuming normal operations
[Mon Feb 25 09:30:46.239091 2019] [core:notice] [pid 14522:tid 140027025042368] AH00094: Command line: ‘/usr/sbin/apache2’
[Mon Feb 25 09:33:53.789749 2019] [mpm_event:notice] [pid 14522:tid 140027025042368] AH00491: caught SIGTERM, shutting down
[Mon Feb 25 09:41:13.835725 2019] [mpm_event:notice] [pid 14714:tid 139636331256768] AH00489: Apache/2.4.29 (Ubuntu) configured – resuming normal operations
[Mon Feb 25 09:41:13.836025 2019] [core:notice] [pid 14714:tid 139636331256768] AH00094: Command line: ‘/usr/sbin/apache2’
[Mon Feb 25 09:49:07.029017 2019] [mpm_event:notice] [pid 14714:tid 139636331256768] AH00491: caught SIGTERM, shutting down
[Mon Feb 25 10:28:43.012275 2019] [mpm_event:notice] [pid 15046:tid 140315883342784] AH00489: Apache/2.4.29 (Ubuntu) configured – resuming normal operations
[Mon Feb 25 10:28:43.031091 2019] [core:notice] [pid 15046:tid 140315883342784] AH00094: Command line: ‘/usr/sbin/apache2’

and access:

MYIP- - [25/Feb/2019:09:23:55 +0000] “GET /HNAP1 HTTP/1.1” 404 477 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “GET /favicon.ico HTTP/1.1” 404 483 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:23:55 +0000] “OPTIONS / HTTP/1.1” 200 181 “-” “Mozilla/5.0 (compatible; Nmap Scripting Engine; Chapter 9. Nmap Scripting Engine | Nmap Network Scanning)”
MYIP- - [25/Feb/2019:09:45:17 +0000] “GET / HTTP/1.1” 200 3477 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36”

That means the most basic of basics isn’t working: your Apache isn’t listening on port 443, meaning the configuration hasn’t activated correctly. I say that, assuming that you still have the firewall disabled, because you wrote:

I have disabled the firewall for the moment.

Now… on the box itself, do you see 443 waiting for connections?

netstat -an | grep ^tcp | grep 443

OK, I have backup (tested working) of the server so can go back if needed.
Yea, firewall is off:

/var/log/apache2$ sudo ufw status
Status: inactive

Running sudo netstat -an | grep ^tcp | grep 443 gives me:

Yeah, that means your Apache isn’t listening on port 443. So the configuration you’ve made is not loading correctly. If it was working, you should have seen an entry for 172.24.228.161:443 LISTEN.

Thanks Tess. I’ve gone back and put the default configs back and still get a error on openssl s_client -connect target:443 and doesn’t return anything. Checked both log file and again not info in either.
Possible bad install of apache2?

I’m going to go back to the back up and have a look at the graylog server dealing with HTTPS.

Thank you for your time and help :slightly_smiling_face:

One important question would of course be: have you prepared Apache so that it can actually serve HTTPS? Apache requires a bunch of additional configuration steps before you can enable HTTPS with TLS/SSL. Not in the least, it’ll require a keypair and a certificate.

No not done any prep to apache, just installed it, I didn’t know that there where other steps required to config SSL. I have the cert and key that we use for sites. Do you have a link to any resources for further config to apache?

Well there you go then :slight_smile: That’s why it’s not working.

Unfortunately, unless your certificate setup involves wildcard certs or a rather crappy CA setup you cannot simply copy keys and certs between hosts. Certificates are made for specific hosts (unless wildcarded), so you’ll have to make sure that they keypair and cert match the host you’re setting up.

There’s plenty of documentation out there on setting up Apache with TLS. The most basic would be the Apache docs themselves (adjust the version number to your current ver):

https://httpd.apache.org/docs/2.4/ssl/ssl_howto.html

1 Like

Thanks Tess, will have a look :slightly_smiling_face:
Yea the certs I’m using are wildcards.

To anyone who finds this post for help

I did fix the problem and here what I did.

So the mistake I made was not enabling the following apache mods

sudo a2enmod ssl
sudo a2enmod proxy
sudo a2enmod rewrite

I also ran

sudo a2enmod proxy_http but is was already enabled
Then restart apache.

My SSL certs where wild card certs (with mean it covers anything with my domain. I’m running this internally, but I still need a cert due to compliance).

I put the certs in a folder called ‘ssl’ in /etc/apache

My 000-default.conf looks like this:

 <VirtualHost *:80>

         ErrorLog ${APACHE_LOG_DIR}/error.log
         CustomLog ${APACHE_LOG_DIR}/access.log combined
         
         #Redirects all traffic from port 80 to port 443
         RewriteEngine on
         RewriteCond %{SERVER_PORT} !^443$
         RewriteRule ^/?(.*)         https://hostname.domain.com/$1 [L,R]
 
</VirtualHost>

This will redirect all traffic to port 80 to 443

My default-ssl.conf looks like this:

<IfModule mod_ssl.c>
        <VirtualHost *:443>

        ServerName hostname.domain.com
        ProxyRequests Off
        ProxyPreserveHost On
        SSLEngine on
        
        #Point to ssl certs
        SSLCertificateFile /etc/apache2/ssl/wildtnewc.cer
        SSLCertificateKeyFile /etc/apache2/ssl/wildnew.key
        SSLCertificateChainFile /etc/apache2/ssl/wildtnewc.cer

                <Proxy *>
                        Order deny,allow
                        Allow from all
                </Proxy>

                <Location />
                        RequestHeader set X-Graylog-Server-URL "https://hostname.domain.com"
                        ProxyPass http://HOSTIP:9000/
                        ProxyPassReverse http://HOSTIP:9000/
                </Location>


                ErrorLog ${APACHE_LOG_DIR}/error.log
                CustomLog ${APACHE_LOG_DIR}/access.log combined


                <FilesMatch "\.(cgi|shtml|phtml|php)$">
                                SSLOptions +StdEnvVars
                </FilesMatch>
                <Directory /usr/lib/cgi-bin>
                                SSLOptions +StdEnvVars
                </Directory>

        </VirtualHost>
</IfModule>

Having ProxyPass and ProxyPassReverse set to 127.0.0.1 did work for me and was getting the following error in the logs

AH01114: HTTP: failed to make connection to backend: 127.0.0.1

So I changed it to my graylogs server IP

Make sure you restart apache after any changes made.Hope this helps :slightly_smiling_face:

1 Like

Hey, great job for getting it to work! Would you be interested in adding your write-up to the Graylog docs? That can be done through Github.

Thanks Tess, Yea definitively, anything to help out :slightly_smiling_face: How do I do it through Github?

1 Like

I would add this to this section:

http://docs.graylog.org/en/3.0/pages/secure/securing.html

What can be found here: https://github.com/Graylog2/documentation/tree/3.0/pages/secure

The readme should guide you to make a work environment: https://github.com/Graylog2/documentation

1 Like

Thanks Jan, will have a look today

1 Like