Output buffer cached almost 300K messages, now processing

For my lab graylog setup i have again seen on rotation of the index (10GB) there are hickups. The index rolled over to graylog_3 and again there are issue noticeable.

This time the output buffer started filling up to +280000 messages to then, at unknown time, start processing messages while now stating “65536 messages in output buffer, 100.00% utilized.”

There are a lot of messages such as per below my-edit-here

2020-10-19T16:09:30+02:00 3c588fa9 / localhost Deflector is pointing to [’’-streamidhere __46], not the newest one: ['streamidhere __47]. Re-pointing.

Stupid but true. There was a ‘no space left on device’ issue which i had solved before.
However, i did not unblock to index from writing. Unless i missed this, it would be neat if this was clearly visible in the UI (that the logs contain READ-ONLY index messages)

Of course, stupidity never arrives alone so i also performed

curl -X PUT “localhost:9200/_all/_settings” -H ‘Content-Type: application/json’ -d’{ “index.blocks.read_only_allow_delete” : false } }’

instead of

curl -X PUT “localhost:9200/_all/_settings” -H ‘Content-Type: application/json’ -d’{ “index.blocks.read_only” : false } }’

which means i deleted all cached message i believe ?

Since i’m using beats and nxlog i assume it may be possible to resend and not duplicate any messages ? is this correct ?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.