How to calculate the increase of no. of Output_processors and Output_batch_size for large incoming data

Graylog Central (peer support)
1

Hi we have Graylog with the following configurations:-

ring_size= 65536 ( # of messages in each buffer)
inputbuffer_ring_size=65536

processbuffer_processors = 5
outputbuffer_processors = 3

output_batch_size = 500
journal_age = 15 min
journal_size = 15 gb

GRAYLOG RAM = 2gb
ELASTICSEARCH RAM =2gb

We are getting like 50,000 messages per second and outputs only 10,000 messages per second
The journal is filling up quickly and getting an error “Journal Utilization is too high” so, can anyone please help me in calculating the increase of output_processors and output_batch_size

2

Did you have multiple machines? How much RAM does the machine have?

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.