Https setup in graylog 3.0

(Mohd Arif) #1

I am setting up https in graylog with CA signed(not self singed) after that i am facing the issue and nothing loads in graylog UI.

Please find my server.log:

is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
root_password_sha2 = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
bin_dir = /usr/share/graylog-server/bin
data_dir = /graylog/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = xxxxxxxxxxxxxxxxxx:9000
http_enable_tls = true
http_tls_cert_file = /graylog/graylog-server/certificates/xxxxxxxxxxxxxxxxxxxxxxx.pem
http_tls_key_file = /graylog/graylog-server/certificates/xxxxxxxxxxxxxxxxxxxxxx_pk8.pem
http_tls_key_password = changeit
elasticsearch_hosts = http://127.0.0.1:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /graylog/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32

I am formatted the key as PKCS#8.pem format and certificate is .pem.
Please let me know if it is really necessary to add the certificate in cacerts?

-Arif

0 Likes

(Jan Doberstein) #2

if you have used your own CA. Did you add this CA to the java keystore that Graylog is able to connect to itself and verify the certificate?

Did you checked the Graylog server.log?

0 Likes

(Mohd Arif) #3

I checked the server.log but did not find anything relevant.
I did not added them in cacert. I will try that now.

0 Likes

(Mohd Arif) #4

I do not have any logs for https but when i set the value of “http_tls_key_password” graylog does not start itself.

2019-03-20T02:32:01.422-04:00 ERROR [ServerBootstrap] Graylog startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[JerseyService [FAILED]]}
at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:741) ~[graylog.jar:?]
at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:553) ~[graylog.jar:?]
at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:314) ~[graylog.jar:?]
at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:148) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:210) [graylog.jar:?]
at org.graylog2.bootstrap.Main.main(Main.java:50) [graylog.jar:?]
Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: JerseyService [FAILED]
Caused by: java.io.IOException: overrun, bytes = 2351
at javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:92) ~[?:1.8.0_201]
at org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69) ~[graylog.jar:?]
at org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:98) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.buildSslEngineConfigurator(JerseyService.java:342) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.startUpApi(JerseyService.java:168) ~[graylog.jar:?]
at org.graylog2.shared.initializers.JerseyService.startUp(JerseyService.java:142) ~[graylog.jar:?]
at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[graylog.jar:?]
at com.google.common.util.concurrent.Callables$4.run(Callables.java:119) ~[graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]

0 Likes

(Jan Doberstein) #5

does your certificate need a password?

0 Likes

(Mohd Arif) #6

I have added my certificate in cacert.jks still no luck.

0 Likes

(Mohd Arif) #7

I have setup the https and everything is working except when i click system–>node–>node ID.
Please find the screenshot…

and when i click node below error message comes…

Could not get plugins

    Getting plugins on node "8cf7f4b5-d19f-469e-b1ab-d71e58334d91" failed: Error: cannot GET https://serverip/api/cluster/8cf7f4b5-d19f-469e-b1ab-d71e58334d91/plugins (500)

    Could not get JVM information

    Getting JVM information for node '8cf7f4b5-d19f-469e-b1ab-d71e58334d91' failed: Error: cannot GET https://serverip:9000/api/cluster/8cf7f4b5-d19f-469e-b1ab-d71e58334d91/jvm (500)

here the graylog server.log

2019-03-28T06:41:05.779-04:00 WARN  [ProxiedResource] Unable to call https://serverip:9000/api/system/metrics/multiple on node <8cf7f4b5-d19f-469e-b1ab-d71e58334d91>
javax.net.ssl.SSLPeerUnverifiedException: Hostname serverip not verified:
 
    subjectAltNames: [my domian]
        at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:329) ~[graylog.jar:?]
        at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:282) ~[graylog.jar:?]
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:167) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) ~[graylog.jar:?]
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) ~[graylog.jar:?]
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at org.graylog2.rest.RemoteInterfaceProvider.lambda$get$0(RemoteInterfaceProvider.java:61) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) ~[graylog.jar:?]
        at okhttp3.RealCall.execute(RealCall.java:77) ~[graylog.jar:?]
        at retrofit2.OkHttpCall.execute(OkHttpCall.java:180) ~[graylog.jar:?]
        at org.graylog2.shared.rest.resources.ProxiedResource.lambda$getForAllNodes$0(ProxiedResource.java:78) ~[graylog.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
0 Likes

(Mohd Arif) #8

I am using CA singed certificates that does not have the ip of Gray log server and even hostname.
it has my domain FQDN and IP which is configured on Apache proxy.

Please share some inputs.

0 Likes

(Jan Doberstein) #9

you have not all needed URIs in your cert…

http_bind_address = xxxxxxxxxxxxxxxxxx:9000

Hostname serverip not verified
0 Likes

(Mohd Arif) #10

Sorry, I did not get.
I need that URI in my certificate or not? if not then why graylog is complaining about the hostname not verified.

0 Likes

(Jan Doberstein) #11

your certificate needs all IPs and URIs that might be used to connect to the URL/Graylog.

0 Likes

(Mohd Arif) #12

I have created the self-signed certificate with graylog server hostname and IP and now this issue resolved.
Thank You @jan

0 Likes

(system) closed #13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

0 Likes