Running pipeline on old messages

Graylog Central (peer support)
1

Hello there! I just discovered the pipeline feature, which allows me to do things like tag messages with a field if they come from certain subnets, which is great! Unfortunately, I haven’t figured out how to get the pipeline processor to run on all the older messages; the fields that I am attempting to use are all already available within the stored messages; I don’t need “raw” information of any kind. Is it possible to “re-run” the pipelines on older data?

If not, is there a way for me to programmatically tag all the old messages that match my rule, essentially manually running the pipeline? Really all it’s doing is running something like:

rule "is_ci_ip"
when
    has_field("http_src") && (cidr_match("<subnet1>", to_ip($message.http_src)) || cidr_match("<subnet2>", to_ip($message.http_src)) || ...)
then
    set_field("is_ci_ip", true);
end

So I could easily do this with an external script, if there were an easy way to set fields. Thanks!

2

he @staticfloat

in Graylog (and many other tools) you manipulate messages on ingest. Once saved no way is given to modify the already messages.

Schema on write VS Schema on read is the keyword for that.

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.