Graylog does not show GelfTCP and GelfUDP logs

Dear all,

I have searched a lot and read almost all the topics which were related to my problem but I could find a way to solve it.
First I found that it is related to Timezone and I fix it both in Linux VMs that I am working with and in my docker-compose file.
Unfortunately, nothing has changed in Graylog UI.
I use tcpdump command and I can see logs in my VM but I can not see them in Graylog.


I really do not know what is the problem because I can see logs with Syslog and it perfectly working.

1 Like

Hi there, so if I understand the problem correctly, you’re running your Graylog deployment in Docker on a couple of Linux VMs, correct? What does your compose file look like? Also, what inputs do you have running in Graylog?

This is my docker-compose file and it is working with Traefik. Inputs of graylog are : GelfTCP - gelfUdp -syslogUDP- syslogTCP.
It is working with Syslog, not Gelf. I also change the timezone of my VMs to Europe.
I also use this command for sending logs from one of my VMs to another one which graylog and traefik are working on it:

echo -n -e ‘{ “version”: “1.1”, “host”: “example.org”, “short_message”: “A short message”, “level”: 5, “_some_info”: “foo” }’"\0" | nc -u -w0 X 12201

Docker-compose file

version: '3.7'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
    image: mongo:4
    volumes:
      - /mnt/Ubuntu/mongo:/data/db
    environment:
      - TZ=Europe/Berlin
    networks:
      - internal
  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.x/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.0
    volumes:
      - elastic:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - TZ=Europe/Berlin
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - internal
  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    image: graylog/graylog:4.0
    volumes:
      - /mnt/Ubuntu/graylog/data:/usr/share/graylog/data
      - /mnt/Ubuntu/graylog/config:/usr/share/graylog/data/config
    environment:
      - GRAYLOG_PASSWORD_SECRET= X
      - GRAYLOG_ROOT_PASSWORD_SHA2= X
      - GRAYLOG_HTTP_EXTERNAL_URI= X
      - TZ=Europe/Berlin
    restart: always
    depends_on:
      - mongodb
      - elasticsearch
    deploy:
      labels:
          #http
        - traefik.http.routers.graylog.service=graylog-svc
        - traefik.http.routers.graylog.rule=Host("")
        - traefik.http.services.graylog-svc.loadbalancer.server.port=9000
        - traefik.http.routers.graylog.entrypoints= web,websecure
         #tls
        - traefik.http.routers.graylog.tls=true
        - traefik.http.routers.graylog.tls.certresolver=le
         #tcp
        - traefik.tcp.routers.graylogtcp.entrypoints= syslogtcp
        - traefik.tcp.routers.graylogtcp.service=graylogtcp-svc
        - traefik.tcp.services.graylogtcp-svc.loadbalancer.server.port=1514
        - traefik.tcp.routers.graylogtcp.rule=HostSNI(`*`)
         #udp
        - traefik.udp.routers.graylogudp.service=graylogudp-svc
        - traefik.udp.routers.graylogudp.entrypoints= syslogudp
        - traefik.udp.services.graylogudp-svc.loadbalancer.server.port=1514/udp
         #GELF TCP
        - traefik.tcp.routers.grayloggelftcp.rule=HostSNI(`*`)
        - traefik.tcp.routers.grayloggelftcp.service=grayloggelftcp-svc
        - traefik.tcp.routers.grayloggelftcp.entrypoints= gelftcp
        - traefik.tcp.services.grayloggelftcp-svc.loadbalancer.server.port=12201
         #GELF UDP
        - traefik.udp.routers.grayloggelfudp.service=grayloggelfudp-svc
        - traefik.udp.routers.grayloggelfudp.entrypoints= gelfudp
        - traefik.udp.services.grayloggelfudp-svc.loadbalancer.server.port=12201/udp
        - traefik.enable=true
        - traefik.docker.network=public
        - traefik.docker.network=internal
        - traefik.http.routers.graylog.middlewares=graylog-header
        - traefik.http.middlewares.graylog-header.headers.customrequestheaders.X-Graylog-Server-URL= X
        - traefik.http.middlewares.graylog-header.headers.customresponseheaders.X-Graylog-Server-URL= X
    # Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
    networks:
      - public
      - internal
networks:
  internal:
    external: true
  public:
    external: true
volumes:
  elastic:
    external: true

and this is my Traefik yml file:

version: '3.3'
services:
  traefik:
    image: traefik:v2.3
    ports:
      - 443:443
      - 80:80
      - 9000:9000
      - 1514:1514
      - 1514:1514/udp
      - 12201:12021
      - 12201:12201/udp
    volumes:
      - letsencrypt-certificates:/letsencrypt-certificates/
    networks:
      - internal
      - public
    command:
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.graylogweb.address=:9000
      - --entrypoints.syslogtcp.address=:1514
      - --entrypoints.syslogudp.address=:1514/udp
      - --entrypoints.gelftcp.address=:12201
      - --entrypoints.gelfudp.address=:12201/udp
      - --providers.docker
      - --providers.docker.swarmMode=true
      - --providers.docker.endpoint=tcp://tasks.docker-socket-proxy:2375 #make secure tasks.[servicename]:[port]
      - --providers.docker.exposedbydefault=false
      - --api
      - --certificatesResolvers.le.acme.email= X
      - --certificatesResolvers.le.acme.storage=/letsencrypt-certificates/acme.json
      - --certificatesResolvers.le.acme.tlsChallenge=true
    deploy:
      labels:
        - traefik.http.middlewares.admin.basicauth.users=X
        - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
        - traefik.http.routers.force-https.rule=hostregexp(`{host:.+}`)
        - traefik.http.routers.force-https.entrypoints=web
        - traefik.http.routers.force-https.middlewares=redirect-to-https
        # Traefik dashboard
        - traefik.http.routers.traefik.rule=Host(" ")
        - traefik.http.routers.traefik.service=api@internal
        - traefik.http.routers.traefik.entrypoints=websecure
        - traefik.http.routers.traefik.tls.certresolver=le
        - traefik.http.routers.traefik.middlewares=admin
        - traefik.enable=true
        - traefik.docker.network=public
        - traefik.docker.network=internal
        - traefik.http.services.traefik.loadbalancer.server.port=8080
  docker-socket-proxy:
    image: tecnativa/docker-socket-proxy
    hostname: docker-socket-proxy
    environment:
      - CONTAINERS=1
      - NETWORKS=1
      - SERVICES=1
      - SWARM=1
      - TASKS=1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - internal
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 500M
      mode: replicated
      replicas: 1
volumes:
  letsencrypt-certificates:
networks:
  internal:
    external: true
  public:
    external: true

Please try to extract also real messages from tcpdump, so we can check if syntax of gelf messages are OK. Also check graylog server log for errors, especially for gelf.