We deployed a graylog 2.3.1 cluster and it runs perfectly until we change the old mongodb instance to a new one.
Recently we deployed a new mongodb replica set and restarted graylog server with new configuration(other components stays the same), then we can’t search any result even if we set time range to “All”. But we queried the elasticsearch with index “graylog*”, there are many results that seems correct.
When we create a new index and a new stream which point to the new index, result can be searched correctly.
Then we checked the mongodb collections and we found that there is no “graylog_0” index in “index_ranges” collection, but there is a default index set in “index_sets” collection and it’s “index_prefix” is “graylog”.
We tried to clear the “index_ranges” collection and restart graylog server but the problem still exists.
Any idea? Is that a bug? Please help!
P.S. There is no error log accrued during the whole time. And we only have one graylog server node.