"Reprocess a stream"

Graylog Central (peer support)
1

OK, not sure if the title makes sense.

Following scenarion. We are migrating from an old Graylog to a new Graylog. Between the two we had to build another Graylog/Elastic cluster because the direct way was not possible. The versions are too far apart.
What we do now is to copy and index from old -> stage -> new. In the new we will reindex the copied index in its final destination. So far so good, this works perfectly.

The problem comes with the Graylog streams in the new cluster. They are not updated an I will not see any old data thru the stream. Our users have only access to the streams and they need to see the old data.

Is it possible to feed an elastic index into an input so the streams get updated?

2

the only option I can see is to update each document with the stream IDs it belongs too.

By far that is not an easy process - and along the way updating your Graylog/Elasticsearch it might be helpful todo the migration of your old Graylog to the new step-by-step and version to version taking your data with you.

3

Hello Jam
is it possible to dump the original message from the index into json and use logstash to feed it back into GrayLog ?

4

nerver tried that … maybe

5

I try now to dump and index _source field with elasticdump (https://www.npmjs.com/package/elasticdump) to a file and feed it into logstash.
Will let you know if this works

6

The extraction of the source log works, but its painful.

I have to reindex the stage index anyhow into the final index, I used elasticsearch to add the streams

curl -s -X POST “x.x.x.x:9200/_reindex?pretty” -H ‘Content-Type: application/json’ -d ’
{
“source”: {
“index”: “test_eventlog_0”
},
“dest”: {
“index”: “test_reindex_0”
},
“script” : {
“inline” : "ctx._source.streams = ["5d317da1cb91842259c81618","5d318a22cb91842259c823b8" ",
“lang” : “painless”
}
}’

With the above command I assign two streams to each log.

7

You can dump the stored message with elasticdump in json.
But it’s not the original.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.