Indices, rotation, disk space

Graylog Central (peer support)
1

Hi,

I have a Graylog server with 500GBs of disk space. I have just 1 index set where all messages go to. I need to keep logs for the last 5 months, they can be deleted after those 5 months. I had issues before as the disk got filled up completely and Graylog stopped working. Don’t want that to happen again so I am looking at message retention for indices, but I am still not sure how to configure it.

This is my current configuration (I did not configure this, the server was handed overt to me and I am trying to set it up properly):

graylog_indices
graylog_indices.png1161×1028 72 KB

(it says 49 years ago but obviously that is not true, the graylog server was built less than a year ago)

How can I achieve my goal (keeping just the last 5 months of logs) so the disk does not get filled up again?

What is the best way of doing what I need without loosing any of the current logs that are newer than 5 months?

Thanks!

2

Now you have rotation period 5 months and keep 20 indexes, it means 20x5=100 months or 8 years.
My suggestion is, make new index set with rotation period 1 month and to keep 5 indexes (or 6 if we want to be sure). Set it as default. After 5 months delete old indexes.

3

To see real time period for index graylog_0 click to Recalculate index ranges.

4

it won’t solve. possible one of the client send epoch 0 time.
I think you got different message if you haven’t recalculate the time range.

5

you have configured the indices to stay for 5 month having 20 of them.

set the indices to daily rotation and keep 155 indices. This gives you granular the option to drop indices if you are short on disk space and you do not loose to much data.

6

Thanks all for your answers.

Can I change the current active write index (graylog_1 in my screenshot) to daily rotation and keep 155 days, without loosing any data newer than 5 months? I would rather not set up a new index but just reconfigure the current one if possible.

7

the deletion is done by count of indices not by the age of messages in the indices. If you tell “keep 155 indices” it will drop the first index after the 156 is created.

8

But if I configure 1 day indices and keep = 155 indices, the large index I currently have (graylog_0) will not be deleted until 155 days have passed from today. I would end up with this large index (containing 5 months of logs) + 1 medium sized index (the currently active one) + 153 daily indices. The disk will fill up before the large index is deleted.

So I assume there is no way for me to do this without loosing some logs on the way. Maybe I should configure 1 day indices and keep = 60 indices for now, then, once the large index is deleted (in 60 days), change the keep parameter to 155 indices (days).

Does this make sense and would it work (I know I will have only 2 months of logs at some point, and then this will grow up to 5 months of logs) ?

Thanks.

9

you can always drop indices manually.

configure 1 day indices and hold 155 of them - monitor your disk space and when you are in a critical area, drop the hudge indices you have manually.

10

Thanks, I’ve done this. Also set up a script that will send me an email when free disk space on the server is below 90% so I can delete the oldest, huge index.

Thanks!

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.