Graylog not receiving any message after disk full/cleanup (Ubuntu 18.04)

[vjcalel]

Hi,

We went out of disk space and performed following steps after stopping both elasticsearch and graylog-server with systemctl:

1. Delete all index and data under /var/lib/graylog-server/journal/messagejournal-0 as journal also got corrupted.

2. Backed up data under /var/lib/elasticsearch/ and then remove all data under it. This freed up 50 GB

3. mongo graylog3 --eval "db.message_counts.remove();" Found here

4. Restart mongod, elasticsearch and graylog-server.

5. Executed this command

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_cluster/settings -d '{ "transient": { "cluster.routing.allocation.disk.threshold_enabled": false } }'

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

Now /var/log/graylog/server.log doesn’t show any error but I am not getting any input messages. Everything shows ok. It is only showing a warning

WARN [AbstractTcpTransport] receiveBufferSize (SO_RCVBUF) for input SyslogTCPInput{title=Syslog-01, type=org.graylog2.inputs.syslog.tcp.SyslogTCPInput, nodeId=7b16810b-b94c-491c-8392-f1688139a9db} (channel [id: 0x57a25ea7, L:/0:0:0:0:0:0:0:0%0:1514]) should be 1048576 but is 2097152

We didn’t put back our old data yet. My intention was to clean all data without deleting settings. It is still using the old input.

What could be the issue ? Plz help.