Docker with datanode issues

Graylog Central (peer support)
1

I tried to use the docker compose from the docs to set up Graylog, but there have been several persistence issues.

The main issue is that the mounts in the compose file do not match those defined in the docker files:
graylog/graylog defines /usr/share/graylog/data, but the composer file suggests three sub directories.
mongo defines /data/configdb, but that is not defined in composer (this is minor)

This causes two anonymous volumes. Recreating the containers will create new volumes instead of attaching the existing volumes.

I updated to use the above volumes.

I also noticed /usr/share/graylog/data/data is empty, but I expected the datanode ca certificate to be there.

Here is my compose file:

name: "graylog"
services:
  mongodb:
    image: "mongo:6.0"
    ports:
      - "27017:27017"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "mongodb_data:/data/db"
      - "mongodb_config:/data/configdb"

  datanode:
    image: "graylog/graylog-datanode:6.1"
    hostname: "datanode"
    environment:
      GRAYLOG_DATANODE_NODE_ID_FILE: "/var/lib/graylog-datanode/node-id"
      GRAYLOG_DATANODE_PASSWORD_SECRET: "<redacted>"
      GRAYLOG_DATANODE_ROOT_PASSWORD_SHA2: "<redacted>"
      GRAYLOG_DATANODE_MONGODB_URI: "mongodb://mongodb:27017/graylog"
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "8999:8999/tcp"   # DataNode API
      - "9200:9200/tcp"
      - "9300:9300/tcp"
    networks:
      - graylog
    volumes:
      - "graylog-datanode:/var/lib/graylog-datanode"
    restart: "on-failure"

  graylog:
    hostname: "server"
    image: "graylog/graylog:6.1"
    depends_on:
      mongodb:
        condition: "service_started"
      datanode:
        condition: "service_started"
    entrypoint: "/usr/bin/tini -- /docker-entrypoint.sh"
    environment:
      GRAYLOG_NODE_ID_FILE: "/usr/share/graylog/data/config/node-id"
      GRAYLOG_HTTP_BIND_ADDRESS: "0.0.0.0:9000"
      GRAYLOG_MONGODB_URI: "mongodb://mongodb:27017/graylog"
      # To make reporting (headless_shell) work inside a Docker container
      GRAYLOG_REPORT_DISABLE_SANDBOX: "true"
      # CHANGE ME (must be at least 16 characters)!
      GRAYLOG_PASSWORD_SECRET: "<redacted>"
      GRAYLOG_ROOT_PASSWORD_SHA2: "<redacted>"
      GRAYLOG_HTTP_EXTERNAL_URI: "https://<hostname>/"
      GRAYLOG_ROOT_EMAIL: "<redacted>"
      GRAYLOG_TRANSPORT_EMAIL_ENABLED: "true"
      GRAYLOG_TRANSPORT_EMAIL_HOSTNAME: <redacted>
      GRAYLOG_TRANSPORT_EMAIL_PORT: 25
      GRAYLOG_TRANSPORT_EMAIL_USE_AUTH: "false"
      GRAYLOG_TRANSPORT_EMAIL_USE_TLS: "true"
      GRAYLOG_TRANSPORT_EMAIL_USE_SSL: "false"
      GRAYLOG_TRANSPORT_EMAIL_SUBJECT_PREFIX: "[<redacted>]"
      GRAYLOG_TRANSPORT_EMAIL_WEB_INTERFACE_URL: "https://<hostname>/"

    ports:
      # Graylog web interface and REST API
      - "127.0.0.1:9000:9000/tcp"
      # Beats
      - "5044:5044/tcp"
      # Syslog TCP
      - "5514:5514/tcp"
      # Syslog UDP
      - "5514:5514/udp"
      # GELF TCP
      - "12201:12201/tcp"
      # GELF UDP
      - "12201:12201/udp"
      # Forwarder data
      - "13301:13301/tcp"
      # Forwarder config
      - "13302:13302/tcp"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "graylog_data:/usr/share/graylog/data"
networks:
  graylog:
    driver: "bridge"
volumes:
  mongodb_data:
  mongodb_config:
  graylog-datanode:
  graylog_data:
2

Hi @murrant, thanks for the hint. The idea was to have three directories for Graylog, since each specific serves different purpose. But you are right, Graylog Docker expects to have everything in graylog/data. We will adjust the docs, since it causes now misunderstanding. And we will fix MongoDB config.

Thank you

1 Like