I know there’s an archiving option with Graylog Enterprise, but how do you do that for a company that doesn’t have the budget?
I read that there is a method of archiving indexes using Curator ( Graylog with curator?) but there are a lot of things I don’t understand.
What is a noop rotation strategy? And is what you really need to do as many conf files as specify here:
And then, my server is almost saturated in disk space, will I have to start a recalculation of all my index ranges?
To repeat my question, I just saw that Curator must be used with a HOT/WARM architecture, however my hard drive is already almost saturated
Can I adapt my architecture without deleting data?
@Cruciani_Elliot I had a similar problem a while ago.
Here’s what I did to solve it:
I was unsuccessful on this issue, so I had to appeal for an “alternative solution” with some “wires” here and there:
I created a step in the “curator” that forces the migration of any index and shard with less than 1 day to the “hot” nodes.
And I created a shell script that applies the settings I need in the newest created index.
It is not a seamless solution, but it solves my problem. I will keep an eye out for new Graylog updates in the hope that this feature I need will appear in new versi…
@reimlima, thank you for the anwser,
what you sent me is going to be very useful to me, however I do not believe that you are talking in the processing of indexes that are already created and that is what may cause me problem since I would like to archive them without deleting them
Yes, this is about created indexes, I just forgot to mention that you have to configure index rotation in your Graylog UI (sorry about that).
To solve the problem I had to choose “
Rotation period: P1D (1 day, a day)” as an option.
You can run all this process manually, so you don’t have to wait until next day to see if it works.
I have a last question: I didn’t understood the action:allocation on your curator config
so, “allocation” is one type of actions Curator can do, it tells Curator to migrate your index from one node to another based on filters you define.
See my configuration as an example:
description: "Apply shard allocation filtering rules to the hot log"
- filtertype: pattern
- filtertype: age
A little of explaining, basically it will migrate indexes named like “graylog_” and older than 1 day from hot nodes to warm nodes.
Curator will know the nodes based on the key “box_type” with value “warm”.
You must tag your nodes using the option in the elasticsearch.yml file:
node.attr.box_type: hot # to hot nodes
node.attr.box_type: warm # to, you guess it, warm nodes
It strongly recomended that you take a moment to read the
Curator’s doc to understand how to configure and run it.
@reimlima, thank you for the anwsers, I’ll try to do that
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.