Struggling with HTTPS & SSL

gmorin · September 21, 2022, 10:20am

Dear members of the Graylog community,

I’m struggling getting SSL & HTTPS to work with Graylog.

My config :

Debian 10 Buster, updated.
Lastest Graylog server (4.3.7) with a free 2Gb/day enterprise license.
Windows ADCS to generate certificates.

I follow these tutorials from the official documentation : Using HTTPS & With ADCS

What I did :

created the graylog.crt and graylog.key files signed with my company’s CA,
generated the ca.crt file from my root CA,
created a folder /etc/graylog/server/certificates and placed the graylog.crt and graylog.key files,
successfully added the graylog.crt cert in the Java Keystore with keytool -importcert -keystore /usr/lib/jvm/java-17-oracle/lib/security/cacerts -alias graylog-ad-signed -file /etc/graylog/server/certificates/graylog.crt
installed the ca.crt with cp ca.crt /usr/local/share/ca-certificates && update-ca-certificates,
restarted Graylog Server.

Then, I modified the server.conf file with :

   - http_bind_address = my-host.domain.lan:9000

   - http_enable_tls = true
   - http_tls_cert_file = /etc/graylog/server/certificates/graylog.crt
   - http_tls_key_file = /etc/graylog/server/certificates/graylog.key
   - http_tls_key_password = password

But, I have no access to the Web UI and some errors are appearing in the logs (see below).
I thought that maybe someone has already encountered this issue and could tell me what I did wrong… The Graylog documentation regarding ADCS issued certs installation was not very clear for me, and I cannot use a self signed cert in this case.

Many thanks in advance for your answers that I hope will be helpful !

The logs :

java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
        at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:229) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:311) [graylog.jar:?]
        at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
        Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
        Caused by: java.security.GeneralSecurityException: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:88) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: org.bouncycastle.operator.OperatorCreationException: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
                at javax.crypto.JceSecurity.getInstance(JceSecurity.java:131) ~[?:?]
                at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
                at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: java.util.jar.JarException: The JCE Provider file:/usr/share/graylog-server/graylog.jar is not signed.
                at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464) ~[?:?]
                at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317) ~[?:?]
                at javax.crypto.JarVerifier.verify(JarVerifier.java:260) ~[?:?]
                at javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130) ~[?:?]
                at javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190) ~[?:?]
                at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218) ~[?:?]
                at javax.crypto.JceSecurity.getInstance(JceSecurity.java:128) ~[?:?]
                at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
                at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
2022-09-21T11:47:08.137+02:00 INFO  [Server] SIGNAL received. Shutting down.
2022-09-21T11:47:08.140+02:00 INFO  [GracefulShutdown] Graceful shutdown initiated.
2022-09-21T11:47:08.141+02:00 INFO  [GracefulShutdown] Node status: [Override lb:DEAD [LB:DEAD]]. Waiting <3sec> for possible load balancers to recognize state change.
2022-09-21T11:47:11.143+02:00 INFO  [GracefulShutdown] Goodbye.```

oebhardt · September 21, 2022, 11:25am

Hi Gabriel,
just to be sure - did You convert the key to PKCS8?

gmorin · September 21, 2022, 11:27am

Hi @oebhardt,
Yes. I converted them from pkcs5 to pkcs8 as mentionned in the documentation

oebhardt · September 21, 2022, 11:38am

Hi Gabriel,
what java-version are you using? It seems there used to be an issue with an older java version:

https://github.com/Graylog2/graylog2-server/issues/9807

gmorin · September 21, 2022, 11:51am

Hi @oebhardt,
Here is my current Java version

user@server:/tmp# java -version
java version "17.0.1" 2021-10-19 LTS
Java(TM) SE Runtime Environment (build 17.0.1+12-LTS-39)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.1+12-LTS-39, mixed mode, sharing)

I recently had to update from OpenJDK 11 to Java 17 because of errors that occured concerning the Office 365 inputs plugin. Changing Java version has solved a lot of errors in the logs.

I didn’t mentioned that I gave up several times on using the HTTPS implementation in the server.conf because it’s the third time that I’m trying to make it work, and still nothing (I’m obviously missing something but what ? ). From what I understood, making HTTPS work using the server.conf method is the only way to get communications between Graylog and agents encrypted, and that’s what I want.

For the moment, in order to get at least the web UI secured, I managed to make a Nginx reverse proxy but I’m not happy with that solution. I think it’s very good for labs but not suited for production use.

oebhardt · September 21, 2022, 12:07pm

Hi @gmorin
I checked my environment (a graylog-cluster using HTTPS); from what I see, You did exactly the same as I did. The only difference is that I did not use a password in my key, but normally that should make no difference at all.
Could you check with “keytool list” if the CA certificates (CA and - if existing - intermediate CA) are correctly imported? On openSUSE, the ca-update-script does that and I would think on debian it does that also, but could you check it?

gmorin · September 21, 2022, 12:19pm

Hi @oebhardt,

Thanks for your fast answer. I was just checking that when I received your answer. Putting the CA certificate in the /usr/local/share/ca-certificates/ and running the update-ca-certificates command returns me that it successfully added the cert.

I can see it when doing a ls /etc/ssl/certs | grep <mycacertname>.
I tried to remove it by doing dpkg-reconfigure ca-certificates and uncheck it from the list, but it does not appear in.
I don’t know if it’s normal or not, because I usually don’t do the cert process by hands.

gmorin · September 21, 2022, 1:07pm

Sorry @oebhardt, I didn’t understood at first what you asked haha.

So, you say that I have to put the CA cert in the java keystore ?

The keytool -list command returns keytool error: java.lang.Exception: Keystore file does not exist: /root/.keystore. I assume that it is not pointing to the good keystore.

I put the Graylog server cert.crt file in the cacert keystore, but not the CA one :

When doing so, I remember having no errors in the output flow.

oebhardt · September 21, 2022, 1:28pm

Hello @gmorin,
I think, we might find the reason here; normally, not the Graylog-Certificate, but the CA-certs should be installed into the CA-keystore, as java should trust all certificates coming from that:

The same can be done if you have already your own certificate authority, you only need the certificates and keys in the format that can be used with Graylog. Add the certificate authority key to the keystore and all certificates that are signed by this certificate authority will be trusted. Same when you pay for certificates or use a free Certificate authority like let’s encrypt to get the server certificates.

Normally, the idea of this store is to trust all certificates signed with the certificates therein. If a selfsigned certificate is used, it is certificate and ca-certificate in one, so it has to be imported.

In this case, the Graylog certificate is not signed by any authority from this file. I am not completely sure (as I did not test it), but I think an imported certificate that is not used to sign any other certificate will not be trusted.
As I mentioned, in SUSE certificates are imported to the Java-Store by the ca-update-script. You should be able to check if they are imported with the command:

keytool -list -cacerts

(Sorry, I forgot the -cacerts the last time.)
If the CA-certificate is not included, it can be imported with the same command You used for the Graylog-certificate.

gmorin · September 21, 2022, 3:10pm

Hi @oebhardt,

Many thanks for your fast answers with all these useful informations.

I successfully added the CA cert file to the cacerts keystore using the command. Sadl it does not work, even after rebooting the server. I really have no idea why…

While trying to reach https://graylog.domain.lan, I get ‘ERR_CONNECTION_REFUSED’ in my browser, and the following lines in the log file :

2022-09-21T16:58:21.543+02:00 INFO  [ImmutableFeatureFlagsCollector] Following feature flags are used: {}
2022-09-21T16:58:22.196+02:00 INFO  [CmdLineTool] Loaded plugin: AWS plugins 4.3.7 [org.graylog.aws.AWSPlugin]
2022-09-21T16:58:22.197+02:00 INFO  [CmdLineTool] Loaded plugin: Enterprise Integrations 4.3.7 [org.graylog.enterprise.integrations.EnterpriseIntegrationsPlugin]
2022-09-21T16:58:22.198+02:00 INFO  [CmdLineTool] Loaded plugin: Integrations 4.3.7 [org.graylog.integrations.IntegrationsPlugin]
2022-09-21T16:58:22.198+02:00 INFO  [CmdLineTool] Loaded plugin: Collector 4.3.7 [org.graylog.plugins.collector.CollectorPlugin]
2022-09-21T16:58:22.199+02:00 INFO  [CmdLineTool] Loaded plugin: Graylog Enterprise 4.3.7 [org.graylog.plugins.enterprise.EnterprisePlugin]
2022-09-21T16:58:22.200+02:00 INFO  [CmdLineTool] Loaded plugin: Graylog Enterprise (ES6 Support) 4.3.7 [org.graylog.plugins.enterprise.org.graylog.plugins.enterprise.es6.EnterpriseES6Plugin]
2022-09-21T16:58:22.200+02:00 INFO  [CmdLineTool] Loaded plugin: Graylog Enterprise (ES7 Support) 4.3.7 [org.graylog.plugins.enterprise.org.graylog.plugins.enterprise.es7.EnterpriseES7Plugin]
2022-09-21T16:58:22.201+02:00 INFO  [CmdLineTool] Loaded plugin: Threat Intelligence Plugin 4.3.7 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2022-09-21T16:58:22.201+02:00 INFO  [CmdLineTool] Loaded plugin: Elasticsearch 6 Support 4.3.7+05bccc7 [org.graylog.storage.elasticsearch6.Elasticsearch6Plugin]
2022-09-21T16:58:22.201+02:00 INFO  [CmdLineTool] Loaded plugin: Elasticsearch 7 Support 4.3.7+05bccc7 [org.graylog.storage.elasticsearch7.Elasticsearch7Plugin]
2022-09-21T16:58:22.229+02:00 INFO  [CmdLineTool] Running with JVM arguments: -Xms4g -Xmx4g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:-OmitStackTraceInFastThrow -Djdk.tls.acknowledgeCloseNotify=true -Dlog4j2.formatMsgNoLookups=true -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=deb
2022-09-21T16:58:22.552+02:00 INFO  [cluster] Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2022-09-21T16:58:22.582+02:00 INFO  [cluster] Cluster description not yet available. Waiting for 30000 ms before timing out
2022-09-21T16:58:22.595+02:00 INFO  [connection] Opened connection [connectionId{localValue:1, serverValue:1137}] to localhost:27017
2022-09-21T16:58:22.598+02:00 INFO  [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 2, 22]}, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=2287309}
2022-09-21T16:58:22.608+02:00 INFO  [connection] Opened connection [connectionId{localValue:2, serverValue:1138}] to localhost:27017
2022-09-21T16:58:22.627+02:00 INFO  [connection] Closed connection [connectionId{localValue:2, serverValue:1138}] to localhost:27017 because the pool has been closed.
2022-09-21T16:58:22.629+02:00 INFO  [MongoDBPreflightCheck] Connected to MongoDB version 4.2.22
2022-09-21T16:58:22.724+02:00 INFO  [SearchDbPreflightCheck] Connected to (Elastic/Open)Search version <Elasticsearch:7.10.2>
2022-09-21T16:58:22.833+02:00 INFO  [Version] HV000001: Hibernate Validator null
2022-09-21T16:58:25.137+02:00 INFO  [InputBufferImpl] Message journal is enabled.
2022-09-21T16:58:25.149+02:00 INFO  [NodeId] Node ID: dd3d775b-c83a-47c0-a277-d3391f195830
2022-09-21T16:58:25.312+02:00 INFO  [LogManager] Loading logs.
2022-09-21T16:58:25.347+02:00 INFO  [LogManager] Logs loading complete.
2022-09-21T16:58:25.349+02:00 INFO  [LocalKafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal
2022-09-21T16:58:25.353+02:00 INFO  [cluster] Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2022-09-21T16:58:25.357+02:00 INFO  [connection] Opened connection [connectionId{localValue:3, serverValue:1139}] to localhost:27017
2022-09-21T16:58:25.358+02:00 INFO  [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 2, 22]}, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=635347}
2022-09-21T16:58:25.367+02:00 INFO  [connection] Opened connection [connectionId{localValue:4, serverValue:1140}] to localhost:27017
2022-09-21T16:58:25.461+02:00 INFO  [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2022-09-21T16:58:25.738+02:00 INFO  [ElasticsearchVersionProvider] Elasticsearch cluster is running Elasticsearch:7.10.2
2022-09-21T16:58:26.334+02:00 INFO  [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2022-09-21T16:58:26.508+02:00 INFO  [connection] Opened connection [connectionId{localValue:5, serverValue:1141}] to localhost:27017
2022-09-21T16:58:26.515+02:00 INFO  [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2022-09-21T16:58:26.998+02:00 INFO  [ServerBootstrap] Graylog server 4.3.7+05bccc7 starting up
2022-09-21T16:58:26.999+02:00 INFO  [ServerBootstrap] JRE: Oracle Corporation 17.0.1 on Linux 4.19.0-21-amd64
2022-09-21T16:58:26.999+02:00 INFO  [ServerBootstrap] Deployment: deb
2022-09-21T16:58:26.999+02:00 INFO  [ServerBootstrap] OS: Debian GNU/Linux 10 (buster) (debian)
2022-09-21T16:58:26.999+02:00 INFO  [ServerBootstrap] Arch: amd64
2022-09-21T16:58:27.107+02:00 INFO  [ServerBootstrap] Running 62 migrations...
2022-09-21T16:58:27.724+02:00 INFO  [PeriodicalsService] Starting 38 periodicals ...
2022-09-21T16:58:27.724+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s].
2022-09-21T16:58:27.730+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
2022-09-21T16:58:27.736+02:00 INFO  [PeriodicalsService] Not starting [org.graylog2.periodical.AlertScannerThread] periodical. Not configured to run on this node.
2022-09-21T16:58:27.736+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2022-09-21T16:58:27.739+02:00 INFO  [LegacyDefaultStreamMigration] Legacy default stream has no connections, no migration needed.
2022-09-21T16:58:27.747+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], polling every [20s].
2022-09-21T16:58:27.755+02:00 INFO  [PeriodicalsService] Not starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. Not configured to run on this node.
2022-09-21T16:58:27.756+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2022-09-21T16:58:27.761+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2022-09-21T16:58:27.771+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexBlockCheck] periodical in [0s], polling every [30s].
2022-09-21T16:58:27.772+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2022-09-21T16:58:27.781+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2022-09-21T16:58:27.794+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2022-09-21T16:58:27.799+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2022-09-21T16:58:27.804+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2022-09-21T16:58:27.851+02:00 INFO  [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2022-09-21T16:58:27.862+02:00 INFO  [connection] Opened connection [connectionId{localValue:7, serverValue:1143}] to localhost:27017
2022-09-21T16:58:27.871+02:00 INFO  [Periodicals] Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2022-09-21T16:58:27.907+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2022-09-21T16:58:27.932+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2022-09-21T16:58:27.944+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@4aa08b26] STARTING
2022-09-21T16:58:27.948+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@4aa08b26] RUNNING
2022-09-21T16:58:27.953+02:00 INFO  [LookupDataAdapterRefreshService] Adding job for <geoip/626902a7f50e1e145300feea/@4aa08b26> [interval=60000ms]
2022-09-21T16:58:27.985+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@4c1a53e0] STARTING
2022-09-21T16:58:27.986+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@4c1a53e0] RUNNING
2022-09-21T16:58:28.013+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.TrafficCounterCalculator] periodical in [0s], polling every [1s].
2022-09-21T16:58:28.014+02:00 INFO  [Periodicals] Starting [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical] periodical in [0s], polling every [1s].
2022-09-21T16:58:28.014+02:00 INFO  [Periodicals] Starting [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp] periodical in [120s], polling every [86400s].
2022-09-21T16:58:28.014+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ESVersionCheckPeriodical] periodical in [0s], polling every [30s].
2022-09-21T16:58:28.014+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.UserSessionTerminationPeriodical] periodical, running forever.
2022-09-21T16:58:28.028+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread] periodical in [0s], polling every [600s].
2022-09-21T16:58:28.028+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads] periodical in [0s], polling every [600s].
2022-09-21T16:58:28.029+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.views.search.db.SearchesCleanUpJob] periodical in [3600s], polling every [28800s].
2022-09-21T16:58:28.043+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@50e8c1e0] STARTING
2022-09-21T16:58:28.047+02:00 INFO  [connection] Opened connection [connectionId{localValue:8, serverValue:1144}] to localhost:27017
2022-09-21T16:58:28.062+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@50e8c1e0] RUNNING
2022-09-21T16:58:28.075+02:00 INFO  [Periodicals] Starting [org.graylog.events.periodicals.EventNotificationStatusCleanUp] periodical in [120s], polling every [86400s].
2022-09-21T16:58:28.076+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].
2022-09-21T16:58:28.076+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.forwarder.ForwarderStatePeriodical] periodical in [0s], polling every [15s].
2022-09-21T16:58:28.076+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.license.LicenseManagerPeriodical] periodical in [0s], polling every [300s].
2022-09-21T16:58:28.076+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.license.LicenseReportPeriodical] periodical in [300s], polling every [3600s].
2022-09-21T16:58:28.076+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.license.StagedLicenseInstallerPeriodical] periodical, running forever.
2022-09-21T16:58:28.092+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.archive.deletion.ArchiveDeletionPeriodical] periodical in [0s], polling every [3600s].
2022-09-21T16:58:28.093+02:00 INFO  [connection] Opened connection [connectionId{localValue:6, serverValue:1142}] to localhost:27017
2022-09-21T16:58:28.103+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.auditlog.mongodb.MongoAuditLogPeriodical] periodical in [0s], polling every [3600s].
2022-09-21T16:58:28.103+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.report.scheduler.ReportPeriodical] periodical in [120s], polling every [60s].
2022-09-21T16:58:28.104+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.files.CleanupPeriodical] periodical in [0s], polling every [86400s].
2022-09-21T16:58:28.119+02:00 INFO  [PeriodicalsService] Not starting [org.graylog.plugins.securityapp.anomaly.retrieval.AnomalyRetrievalPeriodical] periodical. Not configured to run on this node.
2022-09-21T16:58:28.119+02:00 INFO  [PeriodicalsService] Not starting [org.graylog.plugins.securityapp.anomaly.DetectorStatusSyncPeriodical] periodical. Not configured to run on this node.
2022-09-21T16:58:28.170+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@bb02c7f] STARTING
2022-09-21T16:58:28.171+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@bb02c7f] RUNNING
2022-09-21T16:58:28.175+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@55b13e26] STARTING
2022-09-21T16:58:28.176+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@55b13e26] RUNNING
2022-09-21T16:58:28.179+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@58f976f2] STARTING
2022-09-21T16:58:28.192+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@64099285] STARTING
2022-09-21T16:58:28.192+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@64099285] RUNNING
2022-09-21T16:58:28.196+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@58f976f2] RUNNING
2022-09-21T16:58:28.213+02:00 INFO  [LookupTableService] Starting lookup table watchlist/623889b012a6964e938add30 [@325f4fd] using cache watchlist-cache/623889b012a6964e938add2c [@64099285], data adapter watchlist-mongo/623889b012a6964e938add2e [@50e8c1e0]
2022-09-21T16:58:28.213+02:00 INFO  [LookupTableService] Starting lookup table geoip/6269030bf50e1e145300ff5f [@5080f952] using cache geoip/626902e7f50e1e145300ff34 [@58f976f2], data adapter geoip/626902a7f50e1e145300feea [@4aa08b26]
2022-09-21T16:58:28.213+02:00 INFO  [LookupTableService] Starting lookup table whois/627a09bb8c488547a416c5e1 [@47422b33] using cache whois-cache/627a09bb8c488547a416c5df [@bb02c7f], data adapter whois/627a09bb8c488547a416c5dd [@4c1a53e0]
2022-09-21T16:58:31.138+02:00 INFO  [InputSetupService] Triggering launching persisted inputs, node transitioned from UninitializedÂ [LB:DEAD] to FailedÂ [LB:DEAD]
2022-09-21T16:58:31.138+02:00 ERROR [InputSetupService] Not starting any inputs because lifecycle is: FailedÂ [LB:DEAD]
2022-09-21T16:58:31.143+02:00 INFO  [FailureSubmissionQueue] [FailureHandlerService#triggerShutdown] Total number of submitted batches: 0 (0 failures), total number of consumed batches: 0 (0 failures)
2022-09-21T16:58:31.143+02:00 INFO  [FailureHandlingService] The service's thread has been interrupted. The queue currently contains 0 failure batches.
2022-09-21T16:58:31.148+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread].
2022-09-21T16:58:31.148+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ClusterHealthCheckThread].
2022-09-21T16:58:31.148+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexerClusterCheckerThread].
2022-09-21T16:58:31.148+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexBlockCheck].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRetentionThread].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRotationThread].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.VersionCheckThread].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ThrottleStateUpdaterThread].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventPeriodical].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventCleanupPeriodical].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRangesCleanupPeriodical].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.TrafficCounterCalculator].
2022-09-21T16:58:31.149+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ESVersionCheckPeriodical].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.views.search.db.SearchesCleanUpJob].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.events.periodicals.EventNotificationStatusCleanUp].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.forwarder.ForwarderStatePeriodical].
2022-09-21T16:58:31.150+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.license.LicenseManagerPeriodical].
2022-09-21T16:58:31.151+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.license.LicenseReportPeriodical].
2022-09-21T16:58:31.151+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.archive.deletion.ArchiveDeletionPeriodical].
2022-09-21T16:58:31.151+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.auditlog.mongodb.MongoAuditLogPeriodical].
2022-09-21T16:58:31.151+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.report.scheduler.ReportPeriodical].
2022-09-21T16:58:31.152+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.files.CleanupPeriodical].
2022-09-21T16:58:31.148+02:00 INFO  [GracefulShutdownService] Running graceful shutdown for <1> shutdown hooks
2022-09-21T16:58:31.156+02:00 INFO  [GracefulShutdownService] Initiate shutdown for <JobWorkerPool>
2022-09-21T16:58:31.156+02:00 INFO  [JobSchedulerService] Shutdown of job heartbeat executor
2022-09-21T16:58:31.156+02:00 INFO  [GracefulShutdownService] Finished shutdown for <JobWorkerPool>, took 0 ms
2022-09-21T16:58:31.158+02:00 INFO  [LogManager] Shutting down.
2022-09-21T16:58:31.164+02:00 INFO  [LocalKafkaMessageQueueReader] Stopping.
2022-09-21T16:58:31.166+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@58f976f2] STOPPING, was RUNNING
2022-09-21T16:58:31.166+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@58f976f2] TERMINATED, was STOPPING
2022-09-21T16:58:31.176+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@64099285] STOPPING, was RUNNING
2022-09-21T16:58:31.177+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@64099285] TERMINATED, was STOPPING
2022-09-21T16:58:31.178+02:00 INFO  [LogManager] Shutdown complete.
2022-09-21T16:58:31.180+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@55b13e26] STOPPING, was RUNNING
2022-09-21T16:58:31.181+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@55b13e26] TERMINATED, was STOPPING
2022-09-21T16:58:31.187+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@bb02c7f] STOPPING, was RUNNING
2022-09-21T16:58:31.188+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@bb02c7f] TERMINATED, was STOPPING
2022-09-21T16:58:31.191+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@4aa08b26] STOPPING, was RUNNING
2022-09-21T16:58:31.191+02:00 INFO  [LookupDataAdapterRefreshService] Removing job for <geoip/626902a7f50e1e145300feea/@4aa08b26>
2022-09-21T16:58:31.192+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@4aa08b26] TERMINATED, was STOPPING
2022-09-21T16:58:31.196+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@50e8c1e0] STOPPING, was RUNNING
2022-09-21T16:58:31.196+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@50e8c1e0] TERMINATED, was STOPPING
2022-09-21T16:58:31.201+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@4c1a53e0] STOPPING, was RUNNING
2022-09-21T16:58:31.202+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@4c1a53e0] TERMINATED, was STOPPING
2022-09-21T16:58:31.224+02:00 INFO  [LookupDataAdapterRefreshService] Stopping 0 jobs
2022-09-21T16:58:31.236+02:00 INFO  [Buffers] Waiting until [INPUT, PROCESS, OUTPUT] buffers are empty.
2022-09-21T16:58:31.237+02:00 INFO  [Buffers] All buffers are empty. Continuing.
2022-09-21T16:58:31.238+02:00 INFO  [OutputSetupService] Stopping output org.graylog2.outputs.BlockingBatchedESOutput
2022-09-21T16:58:31.934+02:00 INFO  [Server] SIGNAL received. Shutting down.
2022-09-21T16:58:31.938+02:00 INFO  [GracefulShutdown] Graceful shutdown initiated.
2022-09-21T16:58:31.938+02:00 INFO  [GracefulShutdown] Node status: [Override lb:DEADÂ [LB:DEAD]]. Waiting <3sec> for possible load balancers to recognize state change.
2022-09-21T16:58:34.143+02:00 INFO  [FailureHandlingService] Shutting down the service. Processed 0 remaining failure batches.
2022-09-21T16:58:34.143+02:00 INFO  [FailureSubmissionQueue] [FailureHandlerService#shutDown] Total number of submitted batches: 0 (0 failures), total number of consumed batches: 0 (0 failures)
2022-09-21T16:58:34.144+02:00 INFO  [ServiceManagerListener] Services are now stopped.
2022-09-21T16:58:34.144+02:00 ERROR [ServerBootstrap] Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
	at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
	at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
	at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
	at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:229) [graylog.jar:?]
	at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:311) [graylog.jar:?]
	at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
	Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
	Caused by: java.security.GeneralSecurityException: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:88) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: org.bouncycastle.operator.OperatorCreationException: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
		at javax.crypto.JceSecurity.getInstance(JceSecurity.java:131) ~[?:?]
		at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
		at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: java.util.jar.JarException: The JCE Provider file:/usr/share/graylog-server/graylog.jar is not signed.
		at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464) ~[?:?]
		at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317) ~[?:?]
		at javax.crypto.JarVerifier.verify(JarVerifier.java:260) ~[?:?]
		at javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130) ~[?:?]
		at javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190) ~[?:?]
		at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218) ~[?:?]
		at javax.crypto.JceSecurity.getInstance(JceSecurity.java:128) ~[?:?]
		at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
		at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
2022-09-21T16:58:34.941+02:00 INFO  [GracefulShutdown] Goodbye.

gsmith · September 21, 2022, 10:37pm

Hello,
Just chiming in,

Perhaps certs ending in something like this…

http_publish_uri = https://graylog.domain.com:9000/
http_enable_cors = true
http_enable_tls = true
http_tls_cert_file = /etc/ssl/certs/graylog/graylog-certificate.pem <---
http_tls_key_file = /etc/ssl/certs/graylog/graylog-key.pem <---
http_tls_key_password = secret

From the Doc’s here

# Enable HTTPS support for the HTTP interface.
# This secures the communication with the HTTP interface with TLS to prevent request forgery and eavesdropping.
http_enable_tls = true

# The X.509 certificate chain file in PEM format to use for securing the HTTP interface.
http_tls_cert_file = /path/to/graylog-certificate.pem

# The PKCS#8 private key file in PEM format to use for securing the HTTP interface.
http_tls_key_file = /path/to/graylog-key.pem

# The password to unlock the private key used for securing the HTTP interface. (if key is encrypted)
http_tls_key_password = secret

Not sure it will help, just a thought.

gmorin · September 22, 2022, 6:02am

Hi @gsmith,
Well that’s true. I also have the key file in .pem but not the certificate file.
I converted my cert.crt with openssl x509 -in mycert.crt -out mycert.pem -outform PEM and added it.

After I corrected it adding the *.pem files to the path, it does not work better. TI still have no access to my gui… I checked : 9000 is open in the FW, so it’s not a FW related issue.

Here is what I currently have in my server.conf :

[General]
is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret = %secret%
root_password_sha2 = %secret%
root_email = %secret%
root_timezone = Europe/Paris
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin

[HTTP]
http_bind_address = server.domain.tld:9000
http_publish_uri = http://$http_bind_address/


[HTTPS]
http_enable_tls = true
http_tls_cert_file = /etc/graylog/server/certificates/graylog-cert.pem
http_tls_key_file = /etc/graylog/server/certificates/graylog-key.pem
http_tls_key_password = %secret%

[Other]
report_disable_sandbox = true

The other settings are default.

I spotted these errors in my logs :

2022-09-22T08:38:05.324+02:00 WARN  [Indices] Couldn't create index spmhs_132. Error: Unable to create index spmhs_132
org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchException: Unable to create index spmhs_132
	at org.graylog.storage.elasticsearch7.ElasticsearchClient.exceptionFrom(ElasticsearchClient.java:151) ~[?:?]
	at org.graylog.storage.elasticsearch7.ElasticsearchClient.execute(ElasticsearchClient.java:111) ~[?:?]
	at org.graylog.storage.elasticsearch7.IndicesAdapterES7.create(IndicesAdapterES7.java:157) ~[?:?]
	at org.graylog2.indexer.indices.Indices.create(Indices.java:211) ~[graylog.jar:?]
	at org.graylog2.indexer.MongoIndexSet.cycle(MongoIndexSet.java:291) ~[graylog.jar:?]
	at org.graylog2.indexer.rotation.strategies.AbstractRotationStrategy.rotate(AbstractRotationStrategy.java:79) ~[graylog.jar:?]
	at org.graylog2.periodical.IndexRotationThread.checkForRotation(IndexRotationThread.java:113) ~[graylog.jar:?]
	at org.graylog2.periodical.IndexRotationThread.lambda$doRun$0(IndexRotationThread.java:77) ~[graylog.jar:?]
	at java.lang.Iterable.forEach(Iterable.java:75) [?:?]
	at org.graylog2.periodical.IndexRotationThread.doRun(IndexRotationThread.java:73) [graylog.jar:?]
	at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:94) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=validation_exception, reason=Validation Failed: 1: this action would add [4] total shards, but this cluster currently has [1000]/[1000] maximum shards open;]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:187) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1892) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1869) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1626) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1598) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1565) ~[?:?]
	at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.IndicesClient.create(IndicesClient.java:145) ~[?:?]
	at org.graylog.storage.elasticsearch7.IndicesAdapterES7.lambda$create$3(IndicesAdapterES7.java:157) ~[?:?]
	at org.graylog.storage.elasticsearch7.ElasticsearchClient.execute(ElasticsearchClient.java:109) ~[?:?]
	... 15 more
	Suppressed: org.graylog.shaded.elasticsearch7.org.elasticsearch.client.ResponseException: method [PUT], host [http://127.0.0.1:9200], URI [/spmhs_132?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [4] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [4] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"},"status":400}
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.convertResponse(RestClient.java:302) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.performRequest(RestClient.java:272) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.performRequest(RestClient.java:246) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1613) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1598) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1565) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.IndicesClient.create(IndicesClient.java:145) ~[?:?]
		at org.graylog.storage.elasticsearch7.IndicesAdapterES7.lambda$create$3(IndicesAdapterES7.java:157) ~[?:?]
		at org.graylog.storage.elasticsearch7.ElasticsearchClient.execute(ElasticsearchClient.java:109) ~[?:?]
		at org.graylog.storage.elasticsearch7.IndicesAdapterES7.create(IndicesAdapterES7.java:157) ~[?:?]
		at org.graylog2.indexer.indices.Indices.create(Indices.java:211) ~[graylog.jar:?]
		at org.graylog2.indexer.MongoIndexSet.cycle(MongoIndexSet.java:291) ~[graylog.jar:?]
		at org.graylog2.indexer.rotation.strategies.AbstractRotationStrategy.rotate(AbstractRotationStrategy.java:79) ~[graylog.jar:?]
		at org.graylog2.periodical.IndexRotationThread.checkForRotation(IndexRotationThread.java:113) ~[graylog.jar:?]
		at org.graylog2.periodical.IndexRotationThread.lambda$doRun$0(IndexRotationThread.java:77) ~[graylog.jar:?]
		at java.lang.Iterable.forEach(Iterable.java:75) [?:?]
		at org.graylog2.periodical.IndexRotationThread.doRun(IndexRotationThread.java:73) [graylog.jar:?]
		at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:94) [graylog.jar:?]
		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
		at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
		at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]
2022-09-22T08:38:05.327+02:00 ERROR [IndexRotationThread] Couldn't point deflector to a new index
java.lang.RuntimeException: Could not create new target index <spmhs_132>.
	at org.graylog2.indexer.MongoIndexSet.cycle(MongoIndexSet.java:292) ~[graylog.jar:?]
	at org.graylog2.indexer.rotation.strategies.AbstractRotationStrategy.rotate(AbstractRotationStrategy.java:79) ~[graylog.jar:?]
	at org.graylog2.periodical.IndexRotationThread.checkForRotation(IndexRotationThread.java:113) ~[graylog.jar:?]
	at org.graylog2.periodical.IndexRotationThread.lambda$doRun$0(IndexRotationThread.java:77) ~[graylog.jar:?]
	at java.lang.Iterable.forEach(Iterable.java:75) [?:?]
	at org.graylog2.periodical.IndexRotationThread.doRun(IndexRotationThread.java:73) [graylog.jar:?]
	at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:94) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
2022-09-22T08:38:05.395+02:00 INFO  [DeletionRetentionStrategy] Finished index retention strategy [delete] for index <linux__23> in 67ms.
2022-09-22T08:38:07.900+02:00 INFO  [InputSetupService] Triggering launching persisted inputs, node transitioned from UninitializedÂ [LB:DEAD] to FailedÂ [LB:DEAD]
2022-09-22T08:38:07.901+02:00 ERROR [InputSetupService] Not starting any inputs because lifecycle is: FailedÂ [LB:DEAD]
2022-09-22T08:38:07.904+02:00 INFO  [FailureSubmissionQueue] [FailureHandlerService#triggerShutdown] Total number of submitted batches: 0 (0 failures), total number of consumed batches: 0 (0 failures)
2022-09-22T08:38:07.904+02:00 INFO  [FailureHandlingService] The service's thread has been interrupted. The queue currently contains 0 failure batches.
2022-09-22T08:38:07.906+02:00 INFO  [GracefulShutdownService] Running graceful shutdown for <1> shutdown hooks
2022-09-22T08:38:07.906+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread].
2022-09-22T08:38:07.906+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ClusterHealthCheckThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexerClusterCheckerThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexBlockCheck].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRetentionThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRotationThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.VersionCheckThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ThrottleStateUpdaterThread].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventPeriodical].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventCleanupPeriodical].
2022-09-22T08:38:07.907+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.IndexRangesCleanupPeriodical].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.TrafficCounterCalculator].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog2.periodical.ESVersionCheckPeriodical].
2022-09-22T08:38:07.908+02:00 INFO  [LogManager] Shutting down.
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.views.search.db.SearchesCleanUpJob].
2022-09-22T08:38:07.908+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.events.periodicals.EventNotificationStatusCleanUp].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.forwarder.ForwarderStatePeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.license.LicenseManagerPeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.license.LicenseReportPeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.archive.deletion.ArchiveDeletionPeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.auditlog.mongodb.MongoAuditLogPeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.report.scheduler.ReportPeriodical].
2022-09-22T08:38:07.909+02:00 INFO  [Periodicals] Shutting down periodical [org.graylog.plugins.files.CleanupPeriodical].
2022-09-22T08:38:07.910+02:00 INFO  [GracefulShutdownService] Initiate shutdown for <JobWorkerPool>
2022-09-22T08:38:07.910+02:00 INFO  [JobSchedulerService] Shutdown of job heartbeat executor
2022-09-22T08:38:07.911+02:00 INFO  [GracefulShutdownService] Finished shutdown for <JobWorkerPool>, took 0 ms
2022-09-22T08:38:07.911+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@165fb9b0] STOPPING, was RUNNING
2022-09-22T08:38:07.911+02:00 INFO  [LookupTableService] Cache geoip/626902e7f50e1e145300ff34 [@165fb9b0] TERMINATED, was STOPPING
2022-09-22T08:38:07.915+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@45643a36] STOPPING, was RUNNING
2022-09-22T08:38:07.916+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@6f6b5d88] STOPPING, was RUNNING
2022-09-22T08:38:07.917+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@1925276e] STOPPING, was RUNNING
2022-09-22T08:38:07.918+02:00 INFO  [LookupTableService] Cache watchlist-cache/623889b012a6964e938add2c [@45643a36] TERMINATED, was STOPPING
2022-09-22T08:38:07.918+02:00 INFO  [LookupTableService] Cache threat-intel-uncached-adapters/627a09db8c488547a416c611 [@6f6b5d88] TERMINATED, was STOPPING
2022-09-22T08:38:07.931+02:00 INFO  [LookupTableService] Cache whois-cache/627a09bb8c488547a416c5df [@1925276e] TERMINATED, was STOPPING
2022-09-22T08:38:07.932+02:00 INFO  [LookupDataAdapterRefreshService] Removing job for <geoip/626902a7f50e1e145300feea/@6114a552>
2022-09-22T08:38:07.932+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@6114a552] STOPPING, was RUNNING
2022-09-22T08:38:07.933+02:00 INFO  [LogManager] Shutdown complete.
2022-09-22T08:38:07.934+02:00 INFO  [LookupTableService] Data Adapter geoip/626902a7f50e1e145300feea [@6114a552] TERMINATED, was STOPPING
2022-09-22T08:38:07.934+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@67411eab] STOPPING, was RUNNING
2022-09-22T08:38:07.934+02:00 INFO  [LookupTableService] Data Adapter whois/627a09bb8c488547a416c5dd [@67411eab] TERMINATED, was STOPPING
2022-09-22T08:38:07.934+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@12d45aeb] STOPPING, was RUNNING
2022-09-22T08:38:07.934+02:00 INFO  [LookupTableService] Data Adapter watchlist-mongo/623889b012a6964e938add2e [@12d45aeb] TERMINATED, was STOPPING
2022-09-22T08:38:07.935+02:00 INFO  [LookupDataAdapterRefreshService] Stopping 0 jobs
2022-09-22T08:38:07.967+02:00 INFO  [Buffers] Waiting until [INPUT, PROCESS, OUTPUT] buffers are empty.
2022-09-22T08:38:07.968+02:00 INFO  [Buffers] All buffers are empty. Continuing.
2022-09-22T08:38:07.968+02:00 INFO  [OutputSetupService] Stopping output org.graylog2.outputs.BlockingBatchedESOutput
2022-09-22T08:38:07.995+02:00 INFO  [LocalKafkaMessageQueueReader] Stopping.
2022-09-22T08:38:10.905+02:00 INFO  [FailureHandlingService] Shutting down the service. Processed 0 remaining failure batches.
2022-09-22T08:38:10.905+02:00 INFO  [FailureSubmissionQueue] [FailureHandlerService#shutDown] Total number of submitted batches: 0 (0 failures), total number of consumed batches: 0 (0 failures)
2022-09-22T08:38:10.905+02:00 INFO  [ServiceManagerListener] Services are now stopped.
2022-09-22T08:38:10.905+02:00 ERROR [ServerBootstrap] Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
	at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
	at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
	at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
	at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:229) [graylog.jar:?]
	at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:311) [graylog.jar:?]
	at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
	Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
	Caused by: java.security.GeneralSecurityException: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:88) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: org.bouncycastle.operator.OperatorCreationException: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
		at javax.crypto.JceSecurity.getInstance(JceSecurity.java:131) ~[?:?]
		at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
		at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
	Caused by: java.util.jar.JarException: The JCE Provider file:/usr/share/graylog-server/graylog.jar is not signed.
		at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464) ~[?:?]
		at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317) ~[?:?]
		at javax.crypto.JarVerifier.verify(JarVerifier.java:260) ~[?:?]
		at javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130) ~[?:?]
		at javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190) ~[?:?]
		at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218) ~[?:?]
		at javax.crypto.JceSecurity.getInstance(JceSecurity.java:128) ~[?:?]
		at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
		at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
		at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
		at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
		at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
		at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
		at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
		at java.lang.Thread.run(Thread.java:833) ~[?:?]
2022-09-22T08:38:10.932+02:00 INFO  [Server] SIGNAL received. Shutting down.
2022-09-22T08:38:10.935+02:00 INFO  [GracefulShutdown] Graceful shutdown initiated.
2022-09-22T08:38:10.935+02:00 INFO  [GracefulShutdown] Node status: [Override lb:DEADÂ [LB:DEAD]]. Waiting <3sec> for possible load balancers to recognize state change.
2022-09-22T08:38:13.937+02:00 INFO  [GracefulShutdown] Goodbye.

Maybe they have something to do with Graylog not working ?

oebhardt · September 22, 2022, 8:13am

Hi @gmorin
from what I see it seems Your setup is nearly identical to mine. Could You remove the password from the key? (The origial key, that is - not the PKCS8.) The command would be
openssl rsa -in key.crt -out key.pem
Thereafter, it would have to be converted to pkcs8 again.
Just to be sure - could You have a look into Your certificate, just to be sure it is bas64? (The conversion should have done so, just to be sure.)

gmorin · September 22, 2022, 8:18am

Hi @oebhardt,
Thanks for your fast reply, I’ll try to re-convert the key before noon.

How do I check that the cert is base64 ?

oebhardt · September 22, 2022, 8:21am

Sorry…
if it starts with something like this:

-----BEGIN CERTIFICATE-----
MIIGEzCCA/ugAwIBAgIUTO8FMM6dA8s10nBgccb1cdOzjeMwDQYJKoZIhvcNAQEN

and not some binary characters, it should be base64 (or PEM). (As I wrote - the commands You used should have done this, it’s just to be completely sure.)

gmorin · September 22, 2022, 8:25am

No problem !

So I double-checked and yes, it starts with something like this.

I’m in the process of converting the key again without the password, but I think my .key file is password protected so I don’t know how to generate a non-protected .pem file from it

oebhardt · September 22, 2022, 8:30am

Hello,
that should work with
openssl rsa -in file.key -out file.pem
It should ask You for the password for file.key and write an unencrypted, passwordless key to into file.pem.

gmorin · September 22, 2022, 9:24am

Well, I converted the .key in .pem without password, this part worked but when convertig to pkcs8 it asked for a password so I just pressend enter, hope it’ll work…

I’m trying with this setup !

gmorin · September 22, 2022, 9:31am

Nope, does not work. Still this log :

2022-09-22T11:26:35.305+02:00 ERROR [ServerBootstrap] Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:769) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:581) ~[graylog.jar:?]
        at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:295) ~[graylog.jar:?]
        at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:229) [graylog.jar:?]
        at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:311) [graylog.jar:?]
        at org.graylog2.bootstrap.Main.main(Main.java:45) [graylog.jar:?]
        Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
        Caused by: java.security.GeneralSecurityException: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:88) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: org.bouncycastle.operator.OperatorCreationException: 1.2.840.113549.1.5.13 not available: JCE cannot authenticate the provider BC
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
                at javax.crypto.JceSecurity.getInstance(JceSecurity.java:131) ~[?:?]
                at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
                at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]
        Caused by: java.util.jar.JarException: The JCE Provider file:/usr/share/graylog-server/graylog.jar is not signed.
                at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464) ~[?:?]
                at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317) ~[?:?]
                at javax.crypto.JarVerifier.verify(JarVerifier.java:260) ~[?:?]
                at javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130) ~[?:?]
                at javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190) ~[?:?]
                at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218) ~[?:?]
                at javax.crypto.JceSecurity.getInstance(JceSecurity.java:128) ~[?:?]
                at javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:208) ~[?:?]
                at org.bouncycastle.jcajce.util.NamedJcaJceHelper.createSecretKeyFactory(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.PEMUtilities.generateSecretKeyForPKCS5Scheme2(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder$1.get(Unknown Source) ~[graylog.jar:?]
                at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:68) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.doBuildKeyStore(PemKeyStore.java:99) ~[graylog.jar:?]
                at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:85) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:358) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:177) ~[graylog.jar:?]
                at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:158) ~[graylog.jar:?]
                at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
                at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) ~[graylog.jar:?]
                at java.lang.Thread.run(Thread.java:833) ~[?:?]

oebhardt · September 22, 2022, 9:38am

Sorry to read it…
I’ll have to look again at my setup - but it really seems to be quite the same. I won’t be able to do this until tomorrow.
Did You comment out the password line in server.conf? (Sorry, I forgot to mention this.)

Topic		Replies	Views
HTTPS setup in Graylog 3.0.0 Graylog Central (peer support)	3	1798	April 9, 2019
HTTPS Graylog 3.2 Graylog Central (peer support)	3	337	July 3, 2020
Https problem on Graylog 3.0 Graylog Central (peer support)	7	922	April 26, 2019
SSL with graylog-ctl Graylog Central (peer support)	3	407	June 28, 2018
HTTPS does not work but it also spoils HTTP Graylog Central (peer support)	12	2039	September 6, 2021

Struggling with HTTPS & SSL

Related Topics