Graylog deployed on Docker Stack fails to start

Graylog Central (peer support)
1

Description of your problem

When deploying Graylog using “docker stack deploy” it fails the healthcheck (apparently) if we set the http_publish_uri to anything other than 127.0.0.1, even though it starts successfully it will always fail after 120 seconds and quit, the problem is that this uri is not what we want to set, and the REST service then creates wrong urls…

This also does NOT happen if I use “docker-compose up”, this only happens if I use “docker stack deploy”, with nothing else getting changed. On docker-compose up the service starts succesfully with values other than 127.0.0.1.

Description of steps you’ve taken to attempt to solve the issue

If instead of setting my desired publish uri I set it to 127.0.0.1 the service goes up successfully.

Environmental information

Compose file version 3

Operating system information

Docker version 20.10.8
Deploying using docker stack deploy.

Package versions

graylog/graylog:4.1
docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
mongo:4.2

Mounted configuration file is default values with the exception of passwords and smtp values.

NOTE: For all container-based deployments, please include your full, redacted YAML configuration file

docker-compose.yml 
version: '3'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongo:
    image: mongo:4.2
    deploy:
      mode: replicated
      replicas: 1
    volumes:
      - mongo_data:/data/db
    networks:
      - graylog-network
  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - discovery.type=single-node
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    deploy:
      mode: replicated
      replicas: 1
      resources:
        limits:
          memory: 1g
    networks:
      - graylog-network
  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    image: graylog/graylog:4.1
    deploy:
      mode: replicated
      replicas: 1
    volumes:
      - graylog_data:/usr/share/graylog/data
      - /docker/graylog/config:/usr/share/graylog/data/config
    environment:
      - GRAYLOG_HTTP_EXTERNAL_URI=https://myurl.com/
      - GRAYLOG_HTTP_PUBLISH_URI=https://myurl.com/
    entrypoint: /usr/bin/tini -- wait-for-it elasticsearch:9200 --  /docker-entrypoint.sh
    networks:
      - graylog-network
    depends_on:
      - mongo
      - elasticsearch
    ports:
      # Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
      # Log Ports
      - 5555:5555
      - 5555:5555/udp
networks:
  graylog-network:
    driver: overlay
volumes:
  mongo_data:
    driver: local
  es_data:
    driver: local
  graylog_data:
    driver: local

Starting the stack with:

docker stack deploy --compose-file docker-compose.yml graylog

Edit: I just disabled healthcheck to test and the container stays up, but fails to respond, I’m clueless as to why it fails with “docker stack deploy” but works with “docker-compose up”.

2

Hello && Welcome

Please note I just started using Docker but If you can show your Graylog log’s when this issue is occurring I might be able to give you some information on why that’s happening.

When I configured Graylog I used this command below. Was your install similar?

docker run --name graylog --link mongo --link elasticsearch -p 9000:9000 -p 12201:12201 -p 1514:1514 -e GRAYLOG_HTTP_EXTERNAL_URI="https://172.6.27.152:9000/" -d graylog/graylog:4.1

I noticed you GRAYLOG_HTTP_EXTERNAL_URI doesn’t have port 9000 at the end of your url and your using HTTPS. I assume your using certificates. Was this the same on you other installation?

3

It would probably work with a simple docker run command, but I want and need it to work using the stack and the compose file, so it’s a different scenario.

As I said using docker-compose up which creates the containers on the local docker everything works, simply changing the start method to docker stack deploy to create services breaks it.

Also, yes my certificates are correct.

edit: Also, there’s no errors on the log, it even says services are running, but they’re not (they quit after 2 minutes but accessing the url doesn’t work either, so it’s something weird with stack deploy that I couldn’t figure out yet).

4

Actually I just tried again with docker-compose up and the error also happens, if I set http_publish_uri to anything other than 127.0.0.1 graylog just refuses to start, even if I put the machine IP address…

But with 127.0.0.1 I can even see the web but when accessing the API it gets wrong urls (it creates them as 127.0.0.1)…

5

I kinda managed to get it to work by setting the publish and external https urls to end with :443

Apparently not including the port was the main issue, however, the healthcheck was still failing, by disabling the healthcheck I was able to see the container start, stay up, and the REST API urls were all correct.

So, add the port 443 and disable the included healthcheck (I’ll try a custom healthcheck later).

6

Do you have a proxy in front of your Docker (i.e. nginx)?

7

Yes I do… Not nginx but I do.

8

Event thou I’m not using docker, I have mine set 443 --> 9000

and Graylog is set something like this.

9

Yeah I tried that, it does not work, port 9000 is blocked on my firewall though which does explain why defaulting to 9000 is a problem.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.