Well, couldn’t figure this one out.
I ended up using nginx as a reverse proxy for the encryption and just using http for graylog itself. Also had to change the http external publish url to the nginx listen block.
Not my first choice, but now its https traffic to the server