Unable to run graylog in https + docker

gang89 · December 24, 2019, 9:28am

Hi all,

I am currently able to run my graylog in http configuration. But when I try to add the respective https, I am unable to access the web interface via the https://domain:90. Appreciate any advise here. Thank you.

Symptoms
Graylog service is constantly in “starting” -> “restart” without any meaningful errors in the logs. Logs seems to suggest that te service is starting up well…

2019-12-24 09:25:23,326 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: AWS plugins 3.1.3 [org.graylog.aws.AWSPlugin]
2019-12-24 09:25:23,334 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Collector 3.1.3 [org.graylog.plugins.collector.CollectorPlugin]
2019-12-24 09:25:23,336 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Threat Intelligence Plugin 3.1.3 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2019-12-24 09:25:23,978 INFO : org.graylog2.bootstrap.CmdLineTool - Running with JVM arguments: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j.configurationFile=/usr/share/graylog/data/config/log4j2.xml -Djava.library.path=/usr/share/graylog/lib/sigar/ -Dgraylog2.installation_source=docker
2019-12-24 09:25:24,472 INFO : org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
2019-12-24 09:25:29,727 INFO : org.graylog2.shared.buffers.InputBufferImpl - Message journal is enabled.
2019-12-24 09:25:29,769 INFO : org.graylog2.plugin.system.NodeId - No node ID file found. Generated: 0ce6f7b5-5e8e-43c5-b5ee-5a3ba705bd8c
2019-12-24 09:25:30,211 INFO : kafka.log.LogManager - Loading logs.
2019-12-24 09:25:30,217 INFO : kafka.log.LogManager - Logs loading complete.
2019-12-24 09:25:30,306 INFO : kafka.log.LogManager - Created log for partition [messagejournal,0] in /usr/share/graylog/data/journal with properties {file.delete.delay.ms -> 60000, compact -> false, max.message.bytes -> 104857600, min.insync.replicas -> 1, segment.jitter.ms -> 0, index.interval.bytes -> 4096, min.cleanable.dirty.ratio -> 0.5, unclean.leader.election.enable -> true, retention.bytes -> 5368709120, delete.retention.ms -> 86400000, flush.ms -> 60000, segment.bytes -> 104857600, segment.ms -> 3600000, retention.ms -> 43200000, flush.messages -> 1000000, segment.index.bytes -> 1048576}.
2019-12-24 09:25:30,322 INFO : org.graylog2.shared.journal.KafkaJournal - Initialized Kafka based journal at /usr/share/graylog/data/journal
2019-12-24 09:25:30,367 INFO : org.mongodb.driver.cluster - Cluster created with settings {hosts=[mongo:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
2019-12-24 09:25:30,502 INFO : org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out
2019-12-24 09:25:30,597 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:1, serverValue:16}] to mongo:27017
2019-12-24 09:25:30,611 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=mongo:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 16]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=12843597}
2019-12-24 09:25:30,660 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:2, serverValue:17}] to mongo:27017
2019-12-24 09:25:31,252 INFO : org.graylog2.shared.buffers.InputBufferImpl - Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2019-12-24 09:25:31,753 INFO : io.searchbox.client.AbstractJestClient - Setting server pool to a list of 1 servers: [https://vpc-graylog-4zitdep7uyo2jbvppz3wbhywim.ap-southeast-1.es.amazonaws.com/]
2019-12-24 09:25:31,754 INFO : io.searchbox.client.JestClientFactory - Using multi thread/connection supporting pooling connection manager
2019-12-24 09:25:31,930 INFO : io.searchbox.client.JestClientFactory - Using custom ObjectMapper instance
2019-12-24 09:25:31,931 INFO : io.searchbox.client.JestClientFactory - Node Discovery disabled...
2019-12-24 09:25:31,931 INFO : io.searchbox.client.JestClientFactory - Idle connection reaping disabled...
2019-12-24 09:25:32,252 INFO : org.graylog2.shared.buffers.ProcessBuffer - Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-12-24 09:25:32,754 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-12-24 09:25:32,782 INFO : org.graylog2.buffers.OutputBuffer - Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-12-24 09:25:32,805 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:3, serverValue:18}] to mongo:27017
2019-12-24 09:25:32,884 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-12-24 09:25:32,957 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-12-24 09:25:33,038 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-12-24 09:25:33,108 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-12-24 09:25:34,796 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server 3.1.3+cda805f starting up
2019-12-24 09:25:34,796 INFO : org.graylog2.bootstrap.ServerBootstrap - JRE: Oracle Corporation 1.8.0_232 on Linux 4.9.114-moby
2019-12-24 09:25:34,797 INFO : org.graylog2.bootstrap.ServerBootstrap - Deployment: docker
2019-12-24 09:25:34,800 INFO : org.graylog2.bootstrap.ServerBootstrap - OS: Debian GNU/Linux 10 (buster) (debian)
2019-12-24 09:25:34,800 INFO : org.graylog2.bootstrap.ServerBootstrap - Arch: amd64
2019-12-24 09:25:34,879 INFO : org.graylog2.shared.initializers.PeriodicalsService - Starting 29 periodicals ...
2019-12-24 09:25:34,880 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s].
2019-12-24 09:25:34,925 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
2019-12-24 09:25:34,930 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.AlertScannerThread] periodical. Not configured to run on this node.
2019-12-24 09:25:34,930 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2019-12-24 09:25:34,933 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], polling every [20s].
2019-12-24 09:25:34,934 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2019-12-24 09:25:34,934 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2019-12-24 09:25:34,946 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2019-12-24 09:25:34,988 INFO : org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration - Legacy default stream has no connections, no migration needed.
2019-12-24 09:25:35,002 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:4, serverValue:19}] to mongo:27017
2019-12-24 09:25:35,003 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2019-12-24 09:25:35,011 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2019-12-24 09:25:35,014 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2019-12-24 09:25:35,020 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2019-12-24 09:25:35,047 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:20}] to mongo:27017
2019-12-24 09:25:35,052 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2019-12-24 09:25:35,058 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2019-12-24 09:25:35,070 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2019-12-24 09:25:35,076 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, running forever.
2019-12-24 09:25:35,096 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2019-12-24 09:25:35,114 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not configured to run on this node.
2019-12-24 09:25:35,123 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, running forever.
2019-12-24 09:25:35,148 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running forever.
2019-12-24 09:25:35,160 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running forever.
2019-12-24 09:25:35,173 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.TrafficCounterCalculator] periodical in [0s], polling every [1s].
2019-12-24 09:25:35,180 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical] periodical in [0s], polling every [3600s].
2019-12-24 09:25:35,210 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp] periodical in [120s], polling every [86400s].
2019-12-24 09:25:35,212 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread] periodical in [0s], polling every [600s].
2019-12-24 09:25:35,214 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads] periodical in [0s], polling every [600s].
2019-12-24 09:25:35,222 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.views.search.db.SearchesCleanUpJob] periodical in [0s], polling every [28800s].
2019-12-24 09:25:35,265 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:7, serverValue:22}] to mongo:27017
2019-12-24 09:25:35,269 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:9, serverValue:24}] to mongo:27017
2019-12-24 09:25:35,266 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:10, serverValue:25}] to mongo:27017
2019-12-24 09:25:35,266 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:8, serverValue:23}] to mongo:27017
2019-12-24 09:25:35,265 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:6, serverValue:21}] to mongo:27017
2019-12-24 09:25:35,282 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.events.periodicals.EventNotificationStatusCleanUp] periodical in [120s], polling every [86400s].
2019-12-24 09:25:35,283 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].
2019-12-24 09:25:35,350 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:13, serverValue:28}] to mongo:27017
2019-12-24 09:25:35,380 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:11, serverValue:26}] to mongo:27017
2019-12-24 09:25:35,382 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:12, serverValue:27}] to mongo:27017
2019-12-24 09:25:35,599 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/5e019a9f709759001271dbb8] is now STARTING
2019-12-24 09:25:36,195 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/5e019a9f709759001271dbb8] is now RUNNING
2019-12-24 09:25:36,303 WARN : org.graylog2.inputs.transports.UdpTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=Test, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=null} (channel [id: 0xd16a6120, L:/0.0.0.0:12200]) should be 262144 but is 425984.
2019-12-24 09:25:37,062 INFO : org.graylog2.shared.initializers.JerseyService - Enabling CORS for HTTP endpoint
2019-12-24 09:26:06,438 INFO : org.glassfish.grizzly.http.server.NetworkListener - Started listener bound to [0.0.0.0:9000]
2019-12-24 09:26:06,440 INFO : org.glassfish.grizzly.http.server.HttpServer - [HttpServer] Started.
2019-12-24 09:26:06,440 INFO : org.graylog2.shared.initializers.JerseyService - Started REST API at <0.0.0.0:9000>
2019-12-24 09:26:06,441 INFO : org.graylog2.bootstrap.ServerBootstrap - Services started, startup times in ms: {BufferSynchronizerService [RUNNING]=39, KafkaJournal [RUNNING]=41, OutputSetupService [RUNNING]=41, GracefulShutdownService [RUNNING]=51, InputSetupService [RUNNING]=92, JobSchedulerService [RUNNING]=131, EtagService [RUNNING]=132, JournalReader [RUNNING]=162, ConfigurationEtagService [RUNNING]=212, MongoDBProcessingStatusRecorderService [RUNNING]=230, StreamCacheService [RUNNING]=233, LookupTableService [RUNNING]=234, PeriodicalsService [RUNNING]=496, JerseyService [RUNNING]=31584}
2019-12-24 09:26:06,453 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.
2019-12-24 09:26:06,456 INFO : org.graylog2.shared.initializers.ServiceManagerListener - Services are healthy
2019-12-24 09:26:06,457 INFO : org.graylog2.shared.initializers.InputSetupService - Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2019-12-24 09:26:06,474 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/5dfdaed854542f00128de957] is now STARTING
2019-12-24 09:26:06,481 WARN : org.graylog2.inputs.transports.UdpTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=Catalog.API, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=null} (channel [id: 0x0d3930ab, L:/0.0.0.0:12201]) should be 262144 but is 425984.
2019-12-24 09:26:06,483 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/5dfdaed854542f00128de957] is now RUNNING

below is my docker-compose file

version: '3.4'

services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongo:
    image: mongo:3
    volumes:
      - mongo_db:/data/db
    networks:
     - graylog_network

  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    image: graylog/graylog:3.1
    environment:
      # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
      # Password: admin
      - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
      - GRAYLOG_HTTP_EXTERNAL_URI=https://mywildcarddomain:90/
      - GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
      # Elastic Settings
      - GRAYLOG_ELASTICSEARCH_HOSTS=https://myawsElasticSearchService.com/
      ##Https Settings
      - GRAYLOG_HTTP_ENABLE_TLS=true
      - GRAYLOG_HTTP_TLS_CERT_FILE=/run/secrets/graylogcrt.pem
      - GRAYLOG_HTTP_TLS_KEY_FILE=/run/secrets/graylogkey.pem
      - GRAYLOG_HTTP_TLS_KEY_PASSWORD=Asdqwe123!
    depends_on:
      - mongo
    networks:
      - graylog_network
    secrets:
        - graylogcrt.pem
        - graylogkey.pem

    ports:
      # Graylog web interface and REST API
      - 90:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
      # GELF TCP
      - 12200:12200
      # GELF UDP
      - 12200:12200/udp

# Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
volumes:
  mongo_db:
    external: true

networks:
  graylog_network:
      external: true
secrets:                    
    graylogcrt.pem:
        external: true 
    graylogkey.pem:
        external: true

system · January 7, 2020, 9:28am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Cant get https up and running Graylog Central (peer support)	24	5115	March 7, 2019
Run Graylog with Docker over HTTPS on AWS Graylog Central (peer support)	1	1076	December 3, 2019
Input failed to start after https Graylog Central (peer support)	13	1360	March 26, 2020
Inputs appear as not running Graylog Central (peer support) sidecar , nxlog , nodatanx	24	7859	July 18, 2017
Odd thing happen after enable https Graylog Central (peer support)	9	1511	November 6, 2019

Unable to run graylog in https + docker

Related topics