Reprocess already ingested messages

pae23 · June 1, 2018, 2:51pm

Hi,

I just add new pipeline processors and I need to reprocess all (or the one matching a query) the messages already ingested.

is that possible?

Regards,

jochen · June 1, 2018, 3:11pm

No, that’s currently not possible.

github.com/Graylog2/graylog2-server

Re-classifying messages

opened 09:41AM - 23 Feb 14 UTC

henrikjohansen

feature triaged

In an environment where stuff changes a lot it could be very beneficial to have …the ability to re-classify messages. One very real example is adding devices that already have existing messages in ES to a stream. Another could be the merger of 2 separate streams. A third possibility is the ability to re-run messages through an recently added extractor. I feel that this is a necessity for providing credibility and robustness to stream based analyses, reporting and alerting (since streams are such a vital anchor point for the way Graylog2 works). The notion of long running system jobs already exists in Graylog2 - this would make a nice addon. I am aware of complications, limitations, impacts, bla bla, because of the way ES works but this is rather essential because coherency and correctness are an absolute must for critical environments. A fair compromise could be to expose this as an opt-in feature that requires clicking through 50 "I know I am stupid but I really want to do this anyways and nobody else but me is to blame" buttons and not doing this automagically upon events such as adding or modifying extractors).

github.com/Graylog2/graylog2-server

reindex option for extractors

opened 01:33PM - 09 Oct 14 UTC

tomrade

processing feature triaged

Ive been going over graylog2 recently and couldnt find if their was a reindex op…tion. It would be nice to have with regards to extractors , so newer extractors can be applied to existing data, obviously this would need to be low priority compared to the indexing operations, but this would be really useful. Coming from a Splunk background, to an ELK (kibana) and graylog2 background, the one feature I miss is this.

system · June 15, 2018, 3:11pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Re-process already ingested messages Graylog Central (peer support)	1	345	September 9, 2020
How to get a raw version and a parsed version of the same messages Graylog Central (peer support) pipeline-rules	3	912	March 3, 2022
"Reprocess a stream" Graylog Central (peer support)	7	1055	August 2, 2019
Running pipeline on old messages Graylog Central (peer support)	2	1160	December 25, 2019
Are dropped messages counted as ingested? Graylog Central (peer support)	2	631	May 16, 2019

Reprocess already ingested messages

Related topics