Hello everyone, I collect a very large amount of logs, when I do a search (search in last 5, 10.15 minutes …) I do not receive the logs. (just for information, my server logs are powerful, i.e. it is not a resource problem on the server side) While retrieving data for this widget, the following er…

Hello @Labidi , Based on the messages it looks like either your elasticsearch instance/cluster is at the least not accessible, but also possibly not healthy or not able to keep up (resources) with the amount of data it is receiving. It appears the graylog server is journaling messages more quickly t…

Hello @ttsandrew Thanks for your feedback, ES is reachable by Graylog and is green [ayoub status EA] I need a solution. the problem that I receive a lot of logs

Is ES served up on the same host as graylog? Or is it a separate host? If they are on separate hosts then the messages queueing on Graylog more quickly than they can be indexed by ES could be a problem with the network connection between them. How is performance of Graylog? Do you have any pipeline…

Graylog and elasticsearch in the same host ! I assure you that there is not a network problem, graylog has enough ram 12 GB and 8 vcpu and has large amount of storage…

Ok. So where is the issue? Do you have any pipeline rules? Is the storage able to keep up with both journaling and indexing messages?

now it is working properly. its operation is correct I can load (at least 5 minutes …) and is ok, NO I don’t have a pipeline rules :confused: . of course, I indexed the logs for a period of 4 months, I indexed it per month, but it’s getting heavy. so I indexed it daily, (for a period of 4 months)

If there is no evidence of compute exhaustion, there’s no network issue to worry about, and you don’t have any special processing like a pipeline rule then I would suspect storage performance. You mention that you are processing a lot of messages. If you look at iotop/atop/top (wa column) do you see…

I’ll check the document and get back to you , thanks @ttsandrew

For posterity, this was resolved over in this thread. [image] Graylog blocked Graylog Central (peer support) Hello , Suddenly my graylog is crashed, I cannot receive the palo alto firewall logs or no others Vm linux for my infrastructure - Please help [log blocked] […

Problem elasticsearch

Graylog Central (peer support)

ttsandrew November 18, 2020, 9:59pm 10

For posterity, this was resolved over in this thread.

1 Like

Topic		Replies	Views
Graylog blocked Graylog Central (peer support)	30	5145	December 2, 2020
Input buffer is 100% and the logs are not sent to elasticsearch Graylog Central (peer support)	4	542	January 4, 2020
Process buffer and output buffer are full. Journal over the allowed size. No messages written to elastic Graylog Central (peer support) elastic	3	56	October 18, 2024
Graylog-ES Communications Graylog Central (peer support)	12	1696	June 29, 2017
Graylog and Elasticsearch work but I can't reach web interface Graylog Central (peer support)	4	3917	September 27, 2017

Problem elasticsearch

Related topics