Many Pipelines with few Rules OR Few Pipelines with many Rules?

Assuming an equal number of Pipeline rules, is it better to have:

  • One Pipeline with many Pipeline Rules.
  • Many Pipelines with few Pipeline Rules.

Speaking from an architecture or philosophical point of view. Is there a performance difference either way?

Is there a “Graylog way” to do it?

It really depends, there isnt much overhead in the number of pipelines themselves. What you really want to do is make your pipelines into a funnel, so you ideally put your most “expensive” ie slow rules running against the fewest messages possible. So your first few stages should be routing messages to streams that only have a few pipelines associated to them, so that as few messages as possible will even hit the when statement of those rules. Hopefully that makes sense.

1 Like

Understood. Thanks for your reply!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.