Messages from old indexes are not appearing on search

[gasc]

1. Describe your incident:
We had a problem with mongoDB that made a minor upgrade and stopped to work. We downgraded the mongoDB and it started to work again. The problem is: our Graylog have 3 indices: the default (graylog_X) and two more duck and cow.

All messages from the default index are appearing normally after this, but messages from duck and cow are not, graylog know that message are there but is not exhibiting it:

And OpenSearch is Green

2. Describe your environment:
Kubernetes running Graylog 4.3, Opensearch 1.3.5 and Mongo 5.0.6.

3. What steps have you already taken to try and solve the problem?
Tried to recalculate index range, and checked index status, but i’m kind of out of ideas and I don’t see ERRORS in logs.

Do you have any direction to point?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

[joe.gross]

@gasc, have you rotated the active write index for each of them?

[gasc]

Hi Chris, yes I tried this with one of them, but no success.

[frantz]

Did you try to recalculate index ranges for these index ?

[gasc]

yes i recalculated all indices

[gocoolsg]

Possibilities
your streamid got changed?
Try to check if the stream id is retain as-is from old to new

[gasc]

gocoolsg thanks man, this is the problem.

do you have any ideia how to fix this?

i’ll try to change it directly on mongo in my test environment and check if i can recover it.

[gasc]

I chanted the stream ID from the object directly on MongoDB but messages still not appearing. When I recalculated the index ranges to check if works, Graylog rollback the ID for the former stream.

So I think that Graylog has some reference for the old stream. I just don’t know where.

[gasc]

not the ideal solution, but I will stop with this: instead of migrate old messages to the new stream I recreated the old stream directly on mongo

db.streams.insertOne({
  _id: ObjectId("old_id"),
	//fields
})