Profiling graylog stream

Graylog Central (peer support)
1

I have couple of streams, and I’d like to profile/optimize them. For example, one stream has 2 rules:

  • level:<=4
  • message regex match

I’m not sure how does the graylog apply those rule to the stream. Is it in order? Since level:<=4 is quick int compassion, while regex match is quite an expensive operation comparatively.

Also does adding new rules help in some cases? As in former stream rule, message regex match would be sufficient, however, I have a hunch level:<=4 benefits the performance since it reduces messages needed for regex match by 50 times.

Is it possible to have stream cascade, that is have “Warning stream” for example which will filter about 98% of messages, then other streams working on “warning stream” with more complex rules?

Lastly can I see somewhere how much time each rule/stream takes per message, and how many messages per second are they processing?

2

No. The stream router engine tries to execute “cheap” rules first so that “expensive” rules might not have to be executed at all.

See StreamRouterEngine for implementation details:

Yes, absolutely. If a cheap rule can be used to prevent an expensive (usually regular expression) rule to be executed, you should absolutely add it.

Yes, check the metrics of your Graylog node(s) on the System/Nodes/Metrics page in the web interface.

1 Like
3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.