Implement HTTPs to an working Graylog5-Docker (PaloAlto-Certificates)

Graylog Central (peer support)
,
1

I have a running and working Graylog5 with docker. Now i will securing it with http. The graylog.conf is in a persistent directory so that i can modify them easylie.

Our PaloAlto can generate Certificates. We have create a .pem and a .crt file.

Please explane me: is it enough to put the files to the graylog system and modify the config - or is it neccessary to make the “java private store”-Steps in the article i find with google?

My .pem is following:

-----BEGIN CERTIFICATE-----
MIIDRTCCAi2gAwIBAgIFANaoVpcwDQYJKoZIhvcNAQELBQAwEzERMA8GA1UEAxMI
UGFsb0FsdG8wHhcNMjMwNDA1MTYxMDUxWhcNMzMwNDAyMTYxMDUxWjAeMRwwGgYD
...
aRvPHkvDW8ycnxenvdXjGvLZ6NDnBooXJg==
-----END CERTIFICATE-----
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQIJGZBDQ2nPasCAggA
MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEqBBCcY3h0jkdwniEbys9zTMvGBIIE
...
D4dgdBWnqzIDn38l2ZGez42Dgnji6KgJltcjd5NRei9g
-----END ENCRYPTED PRIVATE KEY-----

My .crt is following:

-----BEGIN CERTIFICATE-----
MIIDRTCCAi2gAwIBAgIFANaoVpcwDQYJKoZIhvcNAQELBQAwEzERMA8GA1UEAxMI
UGFsb0FsdG8wHhcNMjMwNDA1MTYxMDUxWhcNMzMwNDAyMTYxMDUxWjAeMRwwGgYD
...
aRvPHkvDW8ycnxenvdXjGvLZ6NDnBooXJg==
-----END CERTIFICATE-----

The graylog.conf changes are:

http_enable_tls = true
http_tls_cert_file = /usr/share/graylog/data/config/cert_server.crt
http_tls_key_file = /usr/share/graylog/data/config/cert_server.pem
http_tls_key_password = verysecretpasswort
2

Hello @MyKeySoftMK

Once you have certs you can use JAVA default keystore called CACERTS.

keytool -import -trustcacerts -file graylog-certificate.pem -alias graylog.domain.com -keystore cacerts

should be good .BTW JAVA default secert is “changeit” you can adjust the secert by…

keytool -storepasswd -keystore /path/to/java/keystore/cacerts
   Type “changeit”
   New-PASS Type “my_new_secert”

What errors do you see in your logs?

3

I using the Docker-Installation of Graylog5 with persistent Config-Files. In this directory i have put the key & cert File. The paths in the config is adaptet.

Must the Keytool execute in the host enviroment or in the docker subsystem?

4

Hey @MyKeySoftMK

You can use keykool on the sysytem. What I have done in the past was find the version of java being used then use locate for cacerts.

You can also use your own keystore but other configuration will be need as shown here

This was just for dev node, I was testing Graylog Docker out using docker-compose.

Example:

keytool -importcert -keystore /var/lib/docker/overlay2/10232988d366d673dd8b1e4f36b9493a623a216fb45a29b841579a9da74c7673/merged/usr/local/openjdk-11/lib/security/cacerts -storepass changeit -alias graylog_server  -file localhost.pem
5

I have import the .pem file. The Result is:

Diesem Zertifikat vertrauen? [Nein]:  Ja
Zertifikat wurde Keystore hinzugefügt

Then I adapt the http_enable_tls = true and try to restart

The container didn´t start again. Error in log is:

2023-04-13 04:43:27,113 INFO : org.graylog2.shared.initializers.ServiceManagerListener - Services are now stopped.
2023-04-13 04:43:27,117 ERROR: org.graylog2.bootstrap.ServerBootstrap - Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
        at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:273) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:324) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:260) [graylog.jar:?]
        at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
        Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
6

Hey @MyKeySoftMK

Seams like you have a configuration issue. Can you show your docker-compose file?

7 
version: "3.8"

services:
  mongodb:
    image: "mongo:5.0"
    volumes:
      - "mongodb_data:/data/db"
    restart: "on-failure"

  opensearch:
    image: "opensearchproject/opensearch:2.4.0"
    environment:
      - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
      - "bootstrap.memory_lock=true"
      - "discovery.type=single-node"
      - "action.auto_create_index=false"
      - "plugins.security.ssl.http.enabled=false"
      - "plugins.security.disabled=true"
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - "os_data:/usr/share/opensearch/data"
    restart: "on-failure"

  graylog:
    hostname: "server"
    image: "${GRAYLOG_IMAGE:-graylog/graylog-enterprise:5.0}"
    depends_on:
      opensearch:
        condition: "service_started"
      mongodb:
        condition: "service_started"
    entrypoint: "/usr/bin/tini -- wait-for-it opensearch:9200 --  /docker-entrypoint.sh"
    ports:
    - "5044:5044/tcp"   # Beats
    - "5140:5140/udp"   # Syslog
    - "5140:5140/tcp"   # Syslog
    - "5555:5555/tcp"   # RAW TCP
    - "5555:5555/udp"   # RAW TCP
    - "9000:9000/tcp"   # Server API
    #- "443:9000/tcp"    # Server API HTTPS
    - "12201:12201/tcp" # GELF TCP
    - "12201:12201/udp" # GELF UDP
    #- "10000:10000/tcp" # Custom TCP port
    #- "10000:10000/udp" # Custom UDP port
    - "13301:13301/tcp" # Forwarder data
    - "13302:13302/tcp" # Forwarder config
    volumes:
      - "graylog_data:/usr/share/graylog/data/data"
      - "graylog_journal:/usr/share/graylog/data/journal"
      - ./config:/usr/share/graylog/data/config
    restart: "on-failure"

volumes:
  mongodb_data:
  os_data:
  graylog_data:
  graylog_journal:
8

Hey @MyKeySoftMK
I see now, your using your graylog.conf file instead
Maybe this might help

Here is my lab Graylgo config

volumes:
      - graylog_journal:/usr/share/graylog/data/journal
      - graylog_bin:/usr/share/graylog/bin
      - graylog_data:/usr/share/graylog/data
environment:
      # Container time Zone
      - TZ=America/Chicago
      # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET=pJod1TRZAckHmqM2oQPqX1qnLVJS99jHm2DuCux2Bpiuu2XLT
      # Password: admin
      - GRAYLOG_ROOT_PASSWORD_SHA2=ef92b778bafe771e892911881f383d4473e94f
      - GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
      - GRAYLOG_HTTP_EXTERNAL_URI=http://192.168.1.28:9000/
      - GRAYLOG_ROOT_TIMEZONE=America/Chicago
      - GRAYLOG_ROOT_EMAIL=greg.smith@domain.com
      - GRAYLOG_HTTP_PUBLISH_URI=http://192.168.1.28:9000/
      - GRAYLOG_TRANSPORT_EMAIL_PROTOCOL=smtp
      - GRAYLOG_HTTP_ENABLE_CORS=true
      - GRAYLOG_TRANSPORT_EMAIL_WEB_INTERFACE_URL=http://192.168.1.28:9000/
      - GRAYLOG_TRANSPORT_EMAIL_HOSTNAME=192.168.1.28
      - GRAYLOG_TRANSPORT_EMAIL_ENABLED=true
      - GRAYLOG_TRANSPORT_EMAIL_PORT=25
      - GRAYLOG_TRANSPORT_EMAIL_USE_AUTH=false
      - GRAYLOG_TRANSPORT_EMAIL_USE_TLS=false
      - GRAYLOG_TRANSPORT_EMAIL_USE_SSL=false
      - GRAYLOG_TRANSPORT_FROM_EMAIL=root@localhost
      - GRAYLOG_TRANSPORT_SUBJECT_PREFIX=[graylog]
      - GRAYLOG_REPORT_DISABLE_SANDBOX=true

I think your issue might be in this part of you configuration, not 100% sure.

      - GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
      - GRAYLOG_HTTP_EXTERNAL_URI=http://192.168.1.28:9000/

What Im not seeing is your network???

Example:

version: '2'
services:
   # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: mongo:4
    network_mode: bridge
   # DB in share for persistence
    volumes:
      - mongo_data:/data/db
   # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
    network_mode: bridge

I used network_mode: bridge setting works great.

And last

    links:
      - mongodb:mongo
      - elasticsearch
    depends_on:
      - mongodb
      - elasticsearch

Hope that helps

9

Thanks for your reply - but i see you are not using https for docker. When i disable https then it works

Here the full error messag

2023-04-13 05:13:14,974 ERROR: org.graylog2.bootstrap.ServerBootstrap - Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
        at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:273) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:324) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:260) [graylog.jar:?]
        at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
        Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
        Caused by: java.security.GeneralSecurityException: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: Error finalising cipher
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:88) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:360) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:178) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:159) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Unknown Source) ~[?:?]
        Caused by: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: Error finalising cipher
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:360) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:178) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:159) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Unknown Source) ~[?:?]
        Caused by: org.bouncycastle.crypto.io.InvalidCipherTextIOException: Error finalising cipher
                at org.bouncycastle.jcajce.io.CipherInputStream.finaliseCipher(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.jcajce.io.CipherInputStream.nextChunk(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.jcajce.io.CipherInputStream.read(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.util.io.Streams.pipeAll(Streams.java:114) ~[graylog.jar:?]
                at org.bouncycastle.util.io.Streams.readAll(Streams.java:41) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:360) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:178) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:159) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Unknown Source) ~[?:?]
        Caused by: javax.crypto.BadPaddingException: pad block corrupted
                at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$BufferedGenericBlockCipher.doFinal(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown Source) ~[graylog.jar:?]
                at javax.crypto.Cipher.doFinal(Unknown Source) ~[?:?]
                at org.bouncycastle.jcajce.io.CipherInputStream.finaliseCipher(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.jcajce.io.CipherInputStream.nextChunk(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.jcajce.io.CipherInputStream.read(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.util.io.Streams.pipeAll(Streams.java:114) ~[graylog.jar:?]
                at org.bouncycastle.util.io.Streams.readAll(Streams.java:41) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:360) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:178) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:159) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Unknown Source) ~[?:?]

I think something with the pem-File is wrong or missconfigured

10

Hey @MyKeySoftMK

Yeah something does look incorrect, either with the configurations for HTTP/TLS or the certs.
I have configured my Graylog Docker config file (i.e.graylog.conf) to have these settings configured.

http_publish_uri = https://graylog.domain.com:9000/
http_enable_tls = true 
http_tls_key_password = secret
 
## I used the directory where i create the certs and chown graylog:graylog -R.

http_tls_key_file = /path/to/certs/graylog-key.pem
 
http_tls_cert_file = /path/to/certs/graylog-certificate.pem

You can also use the environment variables in your Docker-Compose file if you wish.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.