I have graylog cluster (was with 2 nodes but now just with one). A LB ip at the front with public IP which is the one that I use for accesing graylog installed on node.
Now I´m supposed to add certificates and make site secure. I already have wildcards for mydomian.com .How can Use this wildcard for graylog? Should I install a nginx on graylog nod and use it as reverse proxy? Is this the only thing I need to do? any other configuration to add?.
thanks!
So I was able to use nginx as reverse proxy for graylog with tls certificates and now I´m logged in in the page with domain name instead of public IP. however I´m getting an issue when I wanted to start inputs on page related to certificates:
I can´t start inputs that were already running. I´m having this messages in logs:
ion: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_222]
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_222]
at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_222]
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_222]
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_222]
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_222]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_222]
should I configure certificates also in graylog? should I be ussing sames as used in nginx configuration?. I have this configuration now in graylog: ow can I solve this issue?
# GRAYLOG CONFIGURATION FILE
############################
is_master = True
node_id_file = /etc/graylog/server/node-id
password_secret = 2jueVqZpwLLjaWxV
root_username = ptysystem
root_password_sha2 = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxf55c3104a5a78c634ab90bec7b2
root_timezone = EST
plugin_dir = /usr/share/graylog-server/plugin
message_journal_enabled = false
message_journal_dir = /var/lib/graylog-server/journal
###########################
# WEB & REST API
###########################
http_bind_address = 10.0.5.93:9000
http_publish_uri = https://logging.xxxxx.com
http_enable_cors = True
http_enable_gzip = True
http_max_header_size = 8192
http_thread_pool_size = 16
http_enable_tls = False
http_tls_cert_file =
http_tls_key_file =
http_tls_key_password =
#############################
# Elasticsearch
#############################
elasticsearch_hosts = http://10.0.5.93:9200
elasticsearch_connect_timeout = 10s
elasticsearch_socket_timeout = 60s
elasticsearch_max_total_connections = 20
elasticsearch_max_total_connections_per_route = 2
elasticsearch_max_retries = 2
elasticsearch_discovery_enabled = False
elasticsearch_index_prefix = graylog
elasticsearch_template_name = graylog-internal
elasticsearch_analyzer = standard
disable_index_optimization = False
elasticsearch_analyzer = standard
elasticsearch_request_timeout = 1m
index_ranges_cleanup_interval = 1h
output_batch_size = 25
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
outputbuffer_processor_keep_alive_time = 5000
outputbuffer_processor_threads_core_pool_size = 3
outputbuffer_processor_threads_max_pool_size = 30
udp_recvbuffer_sizes = 1048576
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
index_optimization_max_num_segments = 1
no_retention = False
async_eventbus_processors = 2
lb_recognition_period_seconds = 3
lb_throttle_threshold_percentage = 95
stream_processing_timeout = 2000
stream_processing_max_faults = 3
alert_check_interval = 60
output_module_timeout = 10000
stale_master_timeout = 2000
shutdown_timeout = 30000
#################################
# Message Journal
#################################
message_journal_enabled = True
message_journal_dir = /var/lib/graylog-server/journal
message_journal_max_age = 12h
message_journal_max_size = 5gb
message_journal_flush_age = 1m
message_journal_flush_interval = 1000000
message_journal_segment_age = 1h
message_journal_segment_size = 100mb
#############################
# MongoDB
#############################
mongodb_uri = mongodb://graylog:graylog@10.0.5.93:27017/graylog
mongodb_max_connections = 100
mongodb_threads_allowed_to_block_multiplier = 5
#############################
# Script Alerts
#############################
#integrations_web_interface_uri =
integrations_scripts_dir = /usr/share/graylog-server/script
#############################
# Miscellaneous
#############################
rules_file =
http_connect_timeout = 5s
http_read_timeout = 10s
http_write_timeout = 10s
disable_index_optimization = False
index_optimization_max_num_segments = 1
ldap_connection_timeout = 2000
disable_sigar = False
dashboard_widget_default_cache_time = 10s
content_packs_loader_enabled = True
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load =
proxied_requests_thread_pool_size = 32
gc_warning_threshold = 1sas you set the publish uri to the NGINX proxied URL Graylog need to be able to verify the certificate.
Add the certificate to the Java trust store or change the publish_uri to the default and use the (more correct way) http_external_uri for the proxy given address.
So basically need to add line:
http_external_uri = https://logging.xxxxx.com
and set:
http_publish_uri = 127.0.0.1:9000
IS this the best way?
OK so I did the change. At firs I was getting following error on graylog log:
2019-09-16T08:55:15.888-04:00 ERROR [CmdLineTool] Invalid configuration
com.github.joschi.jadconfig.ValidationException: "http_external_uri" must end with a slash ("/")
So I change external uri in graylog with backslah at the end in conf like:
http_external_uri = https://logging.xxxxx.com/
But now page is unaccesible. Is showwing 2 backslashes:
This is my conf right now:
###########################
# WEB & REST API
###########################
http_bind_address = 10.0.5.93:9000
http_publish_uri = http://$http_bind_address/
http_external_uri = https//:logging.xxxx.com/
http_enable_cors = True
http_enable_gzip = True
http_max_header_size = 8192
http_thread_pool_size = 16
http_enable_tls = False
http_tls_cert_file =
http_tls_key_file =
http_tls_key_password =
Should I also configure with backslash at the end of URL on nginx side?
http_publish_uri = http://$http_bind_address/
comment that out will make it work
commented, restarted graylog and nginx and still getting site in blank with 2 backslashes at the end. this is my nginx conf. Nginx is used in another server as a reverse proxy to this graylog server . I just added a new Vhost like this. all other sites that nginx is redirecting are working ok, so configuration is fine as far as I know
if you check the docs … http://docs.graylog.org/en/3.1/pages/configuration/web_interface.html#nginx
you actually see that the proxy_pass does not need a / at the end.
I try with and without and is still not working
Anyway, will keep looking. Don´t know now if this a nginx issue or graylog
what about this configuration? Graylog web interface blank (ubuntu 16.04, followed guide)
Checking graylog configuration doc and it doesn´t talk about this option: web_listen_uri
I´m still getting blank page and no issues in graylog logs…
web_* and rest_* are pre 3.x settings and are not given in Graylog 3.x anymore.
It is not clear for me if you have restarted your browser and anything else or what you have entered in the browser. Means the issue is solved from a logical level - now the really needs to fit that too 
What is the current configuration what did you enter in your browser and what is the result?
I restarted browser, incognito mode used as well. And If I enter URL access:
Is like if it didn´t know what to bring up.
Here latest nginx config comented last line and add the other:
and this is lattest graylog conf:
http_bind_address = 10.0.5.93:9000
http_publish_uri = http://$http_bind_address/
http_external_uri = https://logging.xxxxx.com/
http_enable_cors = True
http_enable_gzip = True
http_max_header_size = 8192
http_thread_pool_size = 16
http_enable_tls = False
http_tls_cert_file =
http_tls_key_file =
http_tls_key_password =
Don´t know what can be the issue now…
ok Now is working. After some nginx reload and graylos service restart. But inputs are not running and in logs I´m getting:
2019-09-17T12:10:46.378-04:00 WARN [ProxiedResource] Unable to call http://api/system/metrics/multiple on node <73d651ad-ad6a-497a-8c3b-23e55b6c5de4>
java.net.UnknownHostException: api
at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[?:1.8.0_222]
at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0_222]
at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0_222]
at okhttp3.Dns$1.lookup(Dns.java:40) ~[graylog.jar:?]
at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185) ~[graylog.jar:?]
at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149) ~[graylog.jar:?]
at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84) ~[graylog.jar:?]
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:214) ~[graylog.jar:?]
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) ~[graylog.jar:?]
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) ~[graylog.jar:?]
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
at org.graylog2.rest.RemoteInterfaceProvider.lambda$get$0(RemoteInterfaceProvider.java:61) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[graylog.jar:?]
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) ~[graylog.jar:?]
at okhttp3.RealCall.execute(RealCall.java:77) ~[graylog.jar:?]
2019-09-17T12:20:01.991-04:00 WARN [ProxiedResource] Unable to call http://api/system/inputstates on node <73d651ad-ad6a-497a-8c3b-23e55b6c5de4>
Should I add: rest_listen_uri ? what else need to add to get inputs running? I know you said is no longer used in graylog 3x but I´m not seying any furhter information to add in configuration of graylog.conf to solve this. Clearly the issue is related to graylog trying to access API. I just wanted to check node infor and get:
once again:
rest_* and web_* settings are PRE 3.x settings. as long as you have 3.x running those settings are useless.
Could you please post as TXT your configuration of Graylog (http_* settings) and your nginx configuration like it is running NOW. but please WITHOUT removing or redacting anything OR redact always in the same way. The current given information are not in a way that anybody other than someone with access to your environment could understand what is wrong.
I already post it before. The only thing that I remove is the domain name. I don´t want to publish that client info. What´s the difference? it could be loggin.anything.com and the issue would still be the same. I´m able to access page and TLS is ok but unable to start running the inputs or gettng node information (rest api issue). However as I was able to see, even thought input not running logs are still ggoing to graylog. Here it is again graylog config:
http_bind_address = 10.0.5.93:9000
http_publish_uri = http://$http_bind_address/
http_external_uri = https://logging.client.com/
http_enable_cors = True
http_enable_gzip = True
http_max_header_size = 8192
http_thread_pool_size = 16
http_enable_tls = False
http_tls_cert_file =
http_tls_key_file =
http_tls_key_password =
and here nginx config:
upstream logging {
server 10.0.5.93:9000 fail_timeout=0;
}
server {
listen 80;
server_name logging.client.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name logging.client.com;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
location / {
client_max_body_size 300M;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect http:// https://;
#proxy_pass http://10.0.5.93:9000/graylog;
proxy_pass http://logging;
}
}
node info:
Change
http_publish_uri = http://$http_bind_address/
to
#http_publish_uri = http://$http_bind_address/
in your Graylog configuration.
( using the variables is not possible in the configuration and only used in the documentation http://docs.graylog.org/en/3.1/pages/configuration/server.conf.html#web-rest-api )
http_bind_address = 10.0.5.93:9000
If you have NGINX on the same host as Graylog change the above to:
http_bind_address = 127.0.0.1:9000
and change the configuration of nginx
upstream logging {
server 127.0.0.1:9000 fail_timeout=0;
}
The nginx is not running on same box. Chek configuration on script. there is a block that wasn´t preformated in my last post. adding here again:
upstream logging {
server 10.0.5.93:9000 fail_timeout=0;
}
So I comment that binding block and inputs are now running finally. Thanks @jan !
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.