Couldn't bulk index - Unable to flush message buffer

Graylog Central (peer support)
1

Hi
I’m using graylog 4.1.6 and I have the error below:

2022-04-12 22:33:50,464 ERROR: org.graylog2.outputs.BlockingBatchedESOutput - Unable to flush message buffer
java.lang.RuntimeException: java.util.concurrent.ExecutionException: ElasticsearchException{message=ElasticsearchException[An error occurred: ]; nested: ElasticsearchStatusException[Elasticsearch exception [type=es_rejected_execution_exception, reason=rejected execution of coordinating operation [coordinating_and_primary_bytes=52127833, replica_bytes=0, all_bytes=52127833, coordinating_operation_bytes=2500996, max_coordinating_and_primary_bytes=53687091]]];, errorDetails=[]}
	at org.graylog2.indexer.messages.Messages.runBulkRequest(Messages.java:235) ~[graylog.jar:?]
	at org.graylog2.indexer.messages.Messages.bulkIndexRequests(Messages.java:151) ~[graylog.jar:?]
	at org.graylog2.indexer.messages.Messages.bulkIndex(Messages.java:143) ~[graylog.jar:?]
	at org.graylog2.indexer.messages.Messages.bulkIndex(Messages.java:123) ~[graylog.jar:?]
	at or

Suppressed: org.graylog.shaded.elasticsearch7.org.elasticsearch.client.ResponseException: method [POST], host [http://elastic-graylog:9200], URI [/_bulk?timeout=1m], status line [HTTP/1.1 429 Too Many Requests]
{"error":{"root_cause":[{"type":"es_rejected_execution_exception","reason":"rejected execution of coordinating operation [coordinating_and_primary_bytes=52127833, replica_bytes=0, all_bytes=52127833, coordinating_operation_bytes=2500996, max_coordinating_and_primary_bytes=53687091]"}],"type":"es_rejected_execution_exception","reason":"rejected execution of coordinating operation [coordinating_and_primary_bytes=52127833, replica_bytes=0, all_bytes=52127833, coordinating_operation_bytes=2500996, max_coordinating_and_primary_bytes=53687091]"},"status":429}
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.convertResponse(RestClient.java:302) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.performRequest(RestClient.java:272) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient.performRequest(RestClient.java:246) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1613) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1583) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1553) ~[?:?]
		at org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:533) ~[?:?]

with this error too

2022-04-12 22:33:50,646 ERROR: org.graylog2.indexer.messages.Messages - Caught exception during bulk indexing: ElasticsearchException{message=ElasticsearchException[An error occurred: ]; nested: ElasticsearchStatusException[Elasticsearch exception [type=es_rejected_execution_exception, reason=rejected execution of coordinating operation [coordinating_and_primary_bytes=52127833, replica_bytes=0, all_bytes=52127833, coordinating_operation_bytes=2514828, max_coordinating_and_primary_bytes=53687091]]];, errorDetails=[]}, retrying (attempt #1).
2022-04-12 22:33:50,646 ERROR: org.graylog2.indexer.messages.Messages - Couldn't bulk index 1000 messages.
java.util.concurrent.ExecutionException: ElasticsearchException{message=ElasticsearchException[An error occurred: ]; nested: ElasticsearchStatusException[Elasticsearch exception [type=es_rejected_execution_exception, reason=rejected execution of coordinating operation [coordinating_and_primary_bytes=52127833, replica_bytes=0, all_bytes=52127833, coordinating_operation_bytes=2514828, max_coordinating_and_primary_bytes=53687091]]];, errorDetails=[]}

Is there someone that can help me to understand what is happeninig?
I add the log file too. Please help me
error-log.pdf (482.1 KB)

2

Hello,

I looked over you logs. I didn’t notice any “connection” issues, mainly bulk request issues.

ERROR: org.graylog2.indexer.messages.Messages - Couldn't bulk index 1000 messages.

What is your Heap for Elasticsearch?

 ERROR: org.graylog2.outputs.BlockingBatchedESOutput - Unable to flush message buffer

Maybe the answer can be found here.

Insure elasticsearch is working correctly.
Perhaps this site might help

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.