Applying rules to multiple streams but keep the data separate

Dear Graylog Ninjas,

As the title suggests I am looking for a way to apply pipeline rules to multiple Streams/Indexes while keeping the messages sorted for each customer.

Lets say I have 50 pipeline rules, each of them check for some kind of abnormality. Failed logins, strange IP addresses, disabled accounts, etc. These pipeline rules are generic and can be applied to any environment. I could create these rules for every stream but this would be very time consuming and prone to typos and what not.

I think the best way to accomplish the goal would be: 1 input, 1 index, 1 stream and 1 pipeline per customer and configure the pipeline rules so that they can be applied to every pipeline that needs them.

What do you guys/girls think? And how would I tell a pipeline rule to push messages to the correct stream without specifying the streamID (without hardcoding it in the rule itself)?

Thank you!

one pipeline can be connected to multiple streams - so you can build one pipeline for this lookup and normalization that is connected to all customer pipelines. Only the routing part needs to have its own rules.

You could check the source stream and if that matches a specific route the message according to that source.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.