WARN ProxiedRessource

Manuu · December 14, 2017, 4:08pm

Hello,

I have some problems configuring graylog and elasticsearch,

Graylog System information is unavailable and jvm 500 couldn’t load histogram data when I check api

My graylog log show this :

WARN [ProxiedResource] Unable to call http://GraylogIP:9000/api/system on node <e???-GraylogNode-???-???f>
java.net.SocketTimeoutException: timeout
at okio.Okio$4.newTimeoutException(Okio.java:230) ~[graylog.jar:?]
at okio.AsyncTimeout.exit(AsyncTimeout.java:285) ~[graylog.jar:?]
at okio.AsyncTimeout$2.read(AsyncTimeout.java:241) ~[graylog.jar:?]
at okio.RealBufferedSource.indexOf(RealBufferedSource.java:345) ~[graylog.jar:?]
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:217) ~[graylog.jar:?]
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:211) ~[graylog.jar:?]
at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189) ~[graylog.jar:?]
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:75) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
at org.graylog2.rest.RemoteInterfaceProvider.lambda$get$0(RemoteInterfaceProvider.java:59) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185) ~[graylog.jar:?]
at okhttp3.RealCall.execute(RealCall.java:69) ~[graylog.jar:?]
at retrofit2.OkHttpCall.execute(OkHttpCall.java:180) ~[graylog.jar:?]
at org.graylog2.shared.rest.resources.ProxiedResource.lambda$getForAllNodes$0(ProxiedResource.java:76) ~[graylog.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.net.SocketException: Socket closed
at java.net.SocketInputStream.read(SocketInputStream.java:204) ~[?:1.8.0_151]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_151]
at okio.Okio$2.read(Okio.java:139) ~[graylog.jar:?]
at okio.AsyncTimeout$2.read(AsyncTimeout.java:237) ~[graylog.jar:?]
… 28 more

Here is my graylog server conf

is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = mypass
root_username = myname
root_password_sha2 = mysha2pass
root_email = mymail
root_timezone = Europe/Paris
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://0.0.0.0:9000/api
rest_enable_cors = true
web_listen_uri = http://0.0.0.0:9000/
elasticsearch_hosts = http://IPelaticsearch:9200, http://IPels:9200, http://IPels:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 1
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = true
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 8
outputbuffer_processors = 5
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32

the other params are commented by “#”

Graylog 2.3.2 is running on CentOS7 with mongodb on the same virtual machine (vsphere)
There is 2 Master node Elasticsearch 5.5.2 and 1 slave.

Tell me if you need more informations,

I’ll be grateful if anyone can help

Regards

jochen · December 15, 2017, 10:46am

Is http://GraylogIP:9000/api/system reachable for the Graylog node itself?
What’s the output of the following command on the machine running Graylog?

# curl -i http://GraylogIP:9000/api/system

Manuu · December 15, 2017, 10:58am

On the browser; http://GraylogIP:9000/api/system is reachable but I have to login with my graylog web ID and Password each time I try to connect

On graylog server the command return :

Thanks a lot

jochen · December 15, 2017, 11:31am

Is there only this one error message in the logs of your Graylog node or is this reproducible?

Manuu · December 15, 2017, 11:41am

I don’t know if it’s reproductible but yes it’s the only error message I have, it happens just after the gelf input started in the log

jochen · December 15, 2017, 11:59am

In that case it’s possible that only this single request timed out and it’s nothing to worry about.

Keep an eye on the logs of your Graylog nodes and open a new topic if you have more questions.

Manuu · December 15, 2017, 12:12pm

After serverals reboot of all my servers, log are still the same,

The point is, I can’t see the Graylog Info

And when I click on the blue text, it shows this

Any ideas?

jan · December 15, 2017, 12:34pm

you might carefully read http://docs.graylog.org/en/2.3/pages/configuration/server.conf.html#general and special the comment for rest_transport_uri:

REST API transport address. Defaults to the value of rest_listen_uri. Exception: If rest_listen_uri is set to a wildcard IP address (0.0.0.0) the first non-loopback IPv4 system address is used.

Manuu · December 15, 2017, 2:40pm

Thanks jan

but, before I try several change on my configuration, no one of my server.conf IP addresses was on 0.0.0.0

I changed rest_transport_uri to http://GraylogIP:9000/api and the problem is still the same.

On the wait to read you,

Regards

jan · December 18, 2017, 7:18am

the configuration you provided to us included wildcards

rest_listen_uri = http://0.0.0.0:9000/api
web_listen_uri = http://0.0.0.0:9000/

Sorry if you modify the provided information in a way that it does not represent the environment, nobody can help you.

Manuu · December 18, 2017, 1:40pm

My apologies, that’s a mistake I should saw,

I correct that, so my last conf is :

web_listen_uri = http://GraylogIP:9000/
rest_listen_uri = http://GraylogIP:9000/api
#rest_transport_uri = not set

The information are still unavailable

Manuu · December 18, 2017, 3:48pm

With API

Is it possible that a bug happened when I converted graylog from virtual box to vmware?

Graylog seems like it can’t connect to itself

jan · December 18, 2017, 3:58pm

@Manuu

you name the problem yourself:

Graylog seems like it can’t connect to itself

now you need to find the reason for that. Did you checked if SELinux or any kind of firewall is the reason?

Manuu · December 18, 2017, 4:00pm

SElinux is removed, and the firewall-cmd disabled

Manuu · December 19, 2017, 1:35pm

I’m trying to solve this issue since 2 weeks, I’m lost now, I don’t know what kind of test I can do or what params I can add…

jan · December 19, 2017, 3:11pm

I did not like to play ping pong with you about all possible reasons why Graylog is not able to connect to itself. You need to find that yourself as we all can only guess the reason, but you are able to check.

Manuu · December 19, 2017, 3:43pm

Yes of course I understand that,

But I was just asking if maybe there is an hidden authentication in the graylog protocol to connect to itself or something else which required attention

system · January 2, 2018, 3:43pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
WARN [ProxiedResource] dont understand Graylog Central (peer support)	1	555	June 26, 2018
ProxiedResource Unable to call http Graylog Central (peer support)	3	2282	March 1, 2019
WARN [ProxiedResource] Unable to call http Graylog Central (peer support)	10	8863	July 27, 2018
System information is currently unavailable Graylog Central (peer support)	6	2328	March 14, 2018
API timeout /system Graylog Central (peer support)	8	2210	February 19, 2022

WARN ProxiedRessource

Related topics