After disk space issue - no out messages - help

(Ryamgithub) #1

Early this morning the messages stopped processing.
I have deleted all of the indexes (except the active one).
I have plenty of disk space now.
I ran the graylog-ctl stop - reboot box,
I see messages coming in but not out.
I am using version Graylog 2.2.3+7adc951 (latest) -

Any suggestion on what to check?

Journal not Getting processed after ES failure
No Logs Collected After Running Out of Diskspace
Journal utilization is too high - elasticsearch running
(Jochen) #2

You might have to delete the Graylog message journal because it might have been corrupted.

(Ryamgithub) #3

So do I just need to shutdown graylog - then delete the journal file - will it get recreated automatically on start?

(Philipp Ruland) #4

Yes it will, as long as the file doesn’t exist and Graylog has write permissions on the folder. :slight_smile:

Greetings, Phil

(Ryamgithub) #5

I see a journal directory in /var/opt/graylog/data is that what I am looking for?

(Ryamgithub) #6

It is a directory - that has a file named
messagejournal-0 (directory)

In the messagejounral-0 directory there are .index and .log files

Which ones should I delete ?

Journal Contains Unprocessed Messages
(Jochen) #7

All of them (while Graylog is stopped).

(Ryamgithub) #8

I apologize for not being clear.

So I need to delete the
graylog-2-commited-read-offset file
messsagejournal-0 directory and all files in that directory

and recovery-point-offset-checkpoint file?

(Philipp Ruland) #9

Delete the complete journal folder :slight_smile:

Aggregation Value Context Error After Full Disk
(Ryamgithub) #10

Thank you.
That resolved that issue
I am back to processing message now!

(system) closed #11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.