Https setup in graylog 3.0

marif · March 20, 2019, 5:49am

I am setting up https in graylog with CA signed(not self singed) after that i am facing the issue and nothing loads in graylog UI.

Please find my server.log:

is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
root_password_sha2 = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
bin_dir = /usr/share/graylog-server/bin
data_dir = /graylog/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = xxxxxxxxxxxxxxxxxx:9000
http_enable_tls = true
http_tls_cert_file = /graylog/graylog-server/certificates/xxxxxxxxxxxxxxxxxxxxxxx.pem
http_tls_key_file = /graylog/graylog-server/certificates/xxxxxxxxxxxxxxxxxxxxxx_pk8.pem
http_tls_key_password = changeit
elasticsearch_hosts = http://127.0.0.1:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /graylog/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32

I am formatted the key as PKCS#8.pem format and certificate is .pem.
Please let me know if it is really necessary to add the certificate in cacerts?

-Arif

jan · March 20, 2019, 8:28am

if you have used your own CA. Did you add this CA to the java keystore that Graylog is able to connect to itself and verify the certificate?

Did you checked the Graylog server.log?

marif · March 20, 2019, 8:57am

I checked the server.log but did not find anything relevant.
I did not added them in cacert. I will try that now.

marif · March 20, 2019, 10:41am

I do not have any logs for https but when i set the value of “http_tls_key_password” graylog does not start itself.

2019-03-20T02:32:01.422-04:00 ERROR [ServerBootstrap] Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:741) ~[graylog.jar:?]
at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:553) ~[graylog.jar:?]
at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:314) ~[graylog.jar:?]
at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:148) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:210) [graylog.jar:?]
at org.graylog2.bootstrap.Main.main(Main.java:50) [graylog.jar:?]
Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
Caused by: java.io.IOException: overrun, bytes = 2351
at javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:92) ~[?:1.8.0_201]
at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69) ~[graylog.jar:?]
at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:98) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:342) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:168) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:142) ~[graylog.jar:?]
at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
at com.google.common.util.concurrent.Callables$4.run(Callables.java:119) ~[graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]

jan · March 20, 2019, 11:13am

does your certificate need a password?

marif · March 22, 2019, 9:21am

I have added my certificate in cacert.jks still no luck.

marif · March 28, 2019, 10:56am

I have setup the https and everything is working except when i click system–>node–>node ID.
Please find the screenshot…

and when i click node below error message comes…

Could not get plugins

    Getting plugins on node "8cf7f4b5-d19f-469e-b1ab-d71e58334d91" failed: Error: cannot GET https://serverip/api/cluster/8cf7f4b5-d19f-469e-b1ab-d71e58334d91/plugins (500)

    Could not get JVM information

    Getting JVM information for node '8cf7f4b5-d19f-469e-b1ab-d71e58334d91' failed: Error: cannot GET https://serverip:9000/api/cluster/8cf7f4b5-d19f-469e-b1ab-d71e58334d91/jvm (500)

here the graylog server.log

2019-03-28T06:41:05.779-04:00 WARN  [ProxiedResource] Unable to call https://serverip:9000/api/system/metrics/multiple on node <8cf7f4b5-d19f-469e-b1ab-d71e58334d91>
javax.net.ssl.SSLPeerUnverifiedException: Hostname serverip not verified:
 
    subjectAltNames: [my domian]
        at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:329) ~[graylog.jar:?]
        at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:282) ~[graylog.jar:?]
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:167) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) ~[graylog.jar:?]
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at org.graylog2.rest.RemoteInterfaceProvider.lambda$get$0(RemoteInterfaceProvider.java:61) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) ~[graylog.jar:?]
        at okhttp3.RealCall.execute(RealCall.java:77) ~[graylog.jar:?]
        at retrofit2.OkHttpCall.execute(OkHttpCall.java:180) ~[graylog.jar:?]
        at org.graylog2.shared.rest.resources.ProxiedResource.lambda$getForAllNodes$0(ProxiedResource.java:78) ~[graylog.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

marif · March 28, 2019, 11:39am

I am using CA singed certificates that does not have the ip of Gray log server and even hostname.
it has my domain FQDN and IP which is configured on Apache proxy.

Please share some inputs.

jan · March 28, 2019, 3:32pm

you have not all needed URIs in your cert…

http_bind_address = xxxxxxxxxxxxxxxxxx:9000

Hostname serverip not verified

marif · March 29, 2019, 5:24am

Sorry, I did not get.
I need that URI in my certificate or not? if not then why graylog is complaining about the hostname not verified.

jan · March 29, 2019, 10:23am

your certificate needs all IPs and URIs that might be used to connect to the URL/Graylog.

marif · April 1, 2019, 10:43am

I have created the self-signed certificate with graylog server hostname and IP and now this issue resolved.
Thank You @jan

system · April 15, 2019, 10:43am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
HTTPS setup in Graylog 3.0.0 Graylog Central (peer support)	3	1890	April 9, 2019
Cant get HTTPS working on graylog Graylog Central (peer support)	12	4646	June 7, 2017
Graylog https Setup Graylog Central (peer support)	20	11325	February 21, 2018
Graylog 3.0.2 https Graylog Central (peer support) sidecar	11	4352	September 2, 2019
Configuration HTTPS Graylog Central (peer support)	17	4180	April 26, 2018

Https setup in graylog 3.0

Related topics