Pipeline rules performance considerations

Graylog Central (peer support)
1

Hello,

I am wondering about some possible performance considerations/recommendations for pipeline rules. I am running some rules with roughly 20,000 events/sec and it could really make a difference.

For example:

Is there a performance impact when making the code readable by adding more lines instead of compact one_line shots?

I mean, is something like:

set_field(“fieldname”, to_long($message.field1) * to_long($message.field2) + 3)

faster than:

let value = to_long($message.field1);

let value2 = to_long($message.field2);

let value3 = value  * value2 + 3;

set_field(“fieldname”, value3);

Or, when setting the values for several fields in a pipeline rule, is it faster to call set_field for each one of the fields? or, is it better to create a map with the values and field names and using set_fields to write them in a single operation?

Some mention of this in the documentation would be great.

Thanks!

2

You have many metrics too observe pipelines performances.

You should try all solutions and check metrics to see which is the faster.