I used to work with graylog a lot since many years. But actually i am not really sure whats the fastest way for routing and sorting messages into streams to process them in separate pipelines.
So the big question for me is it faster to use the stream rules within the Message Filter Chain to route messages into streams or is it more faster to use one pipeline attached to the stream “All messages” with pipeline rules using the “route_to_stream” and “remove_from_stream” functions?
Has someone some serious experiences about that whats the fastest way for stream routing. At the end the goal is clear you want to have “clear” streams to attach them to pipelines which do only message specific processing.
So i think there must be some performance differences but actually i have know answer to the question and a solution for a measurement. The assumption for that case is that you have more than 10k/s messages.