I have a 3 node cluster , all 3 running both graylog and elastrcsearch.Version is 2.2.1
Till week ago I had only about 100 msg/second and onyl the default stream and Default index set.
A week ago I connected a new input with about 5000 msg/sec and for it I set up a new stream SRZ, where only stream rule is simple:
“field source must contain SRZ-1200”
and it is connected with new SRZ index set, and with this option selected:
“Remove matches from ‘All messages’ stream”
(because I wanted all SRZ messages to go to only SRZ stream and to be stored only in new SRZ index set)
It seems to work for streams, as I see large number of msgs in SRZ stream and only around 50 msg/sec in Defauls stream.
But problem is when looking at the indices windows. Both SRZ index and also the Default index seem to be filling approximately at the same pace…many 1000 of msgs per second. It looks like the SRZ messages are stored in both indexes. But when I click on few SRZ messages, they are alwasy show as "stored in srz_),
In troubleshooting this, I changed the destination of SRZ stream to be the Default Index set. This works fine, as now only Default Index set is growing. But I dont want that, I want the SRZ messages to be stored in SRZ index only.