Remove matches from All messages stream

[RochonC (Claude Rochon)]

Why is the option ‘Remove matches from ‘All messages’ stream’ not checked in the 4 OOTB streams: nginx HTTP 404s, nginx HTTP 4XXs, nginx HTTP 5XXs, nginx requests. Does this means that for one input message, it’s now found in All messages AND depending on the message values & rules evaluation, could be found in one of the streams: nginx HTTP 404s, nginx HTTP 4XXs, nginx HTTP 5XXs, nginx requests ?

For example, if the input message contains:
• Field source = nginx
• Field response_status = 404

…will the message be found in 4 streams: All messages, nginx HTTP 404s, nginx HTTP 4XXs, nginx requests ? Is this 4 copies of the same input message ending up in the ‘Default index set’ ?

[jochen]

@RochonC

Does this means that for one input message, it’s now found in All messages AND depending on the message values & rules evaluation, could be found in one of the streams: nginx HTTP 404s, nginx HTTP 4XXs, nginx HTTP 5XXs, nginx requests ?

Yes, exactly.

@RochonC

Is this 4 copies of the same input message ending up in the ‘Default index set’ ?

No.

Please refer to http://docs.graylog.org/en/2.3/pages/streams.html and http://docs.graylog.org/en/2.3/pages/configuration/index_model.html for information about streams and the underlying index model used by Graylog.

[RochonC (Claude Rochon)]

Thanks for your reply. If I got it right, the ‘Remove matches from ‘All messages’ stream’ option becomes handy when the stream(s) where a message end up in, uses a different ‘index set’ than the ‘Default index set’ and we don’t want the message written in both index set.