Solution for a closed topic (log retention)

I open this topic to share a solution of this closed topic(Curator - Different log retention on development server than others).

To do this, I created streams filtered by regular expressions that matches the end of the servers name.
In my case, regex: -(dev|DEV)\d?(.mydomain.name)?$
→ match all server name ending with “-dev” / “-DEV” or “-dev.mydomain.name” / “-DEV.mydomain.name”

Then, to be able to apply retention rule for each, I create an index set with my preferences (rotation period: P1D; strategy: Delete after 31d)

Finally, I edit my action file to make a specific rule for these servers.
/etc/graylog/action/action_hot_warm.yml
3:
action: allocation
description: “Apply the shard allocation filtering rules to the concerned indexes”
options:
key: box_type
value: warm
allocation_type: require
wait_for_completion: true
timeout_override:
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: dev_
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 7
4:
action: forcemerge
description: “Performs a forceMerge on the affected indexes up to the value of ‘max_num_segments’ by shard”
options:
max_num_segments: 1
delay:
timeout_override: 21600
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: dev_
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 8

@Matthias
Nice, and thanks for sharing. If you could format you code that would be great. Makes it easier to read.
Thanks

1 Like

Doesn’t have the “three dots” to edit my post :thinking:
An other way to do it ?

Maybe something on the configuration bar.

https://community.graylog.org/t/community-guidelines/6649#format-markdown

3:
action: allocation
description: “Apply the shard allocation filtering rules to the concerned indexes”
options:
key: box_type
value: warm
allocation_type: require
wait_for_completion: true
timeout_override:
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: dev_
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 7
4:
action: forcemerge
description: “Performs a forceMerge on the affected indexes up to the value of ‘max_num_segments’ by shard”
options:
max_num_segments: 1
delay:
timeout_override: 21600
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: dev_
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 8
1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.