Pipeline is not working for other streams


(Tafsir) #1

Hi All

I have created a Pipeline named Test1 and connected with a newly created stream Logging user activity but it’s showing current throughput 0. Also when I connected the default All messages stream to pipeline Test1 then data start coming.

FYI - Logs is already coming in Logging user activity but in the pipeline, throughput is 0.

It’s working for default All Messages stream but not working for newly created streams.

Here is the Message Processors Configuration order.

Please help me with this.

Thanks


(Jan Doberstein) #2

He @Tafsir_Alam

what is your processing order in system > configuration ? That might be the reason for that.


(Tafsir) #3

@jan Here is the processing order.


(Prakash A) #4

Dear @Tafsir_Alam, I am also facing the same issue. Maybe we will discuss in this forum for further process.


(Jan Doberstein) #5

The routing into streams is done in the “message filter chain” what in your case runs after the processing pipelines.

So if you change the processor configuration order that the processing pipeline runs after the message filter chain you can use the stream rules to route messages into streams and then run the processing pipelines on the data that is in the stream.

currently you would need a processing pipeline that routes the messages into the streams connected to all messages and then a second pipeline that does the extraction based on the data in the streams.


(Tafsir) #6

@jan Is this create any problem on the messages that coming on the graylog if I change the order that is processing pipeline runs after the message filter chain?


(Jan Doberstein) #7

it depends on your processing and can’t be answered in general.


(Tafsir) #8

What Processing @jan ?


(Jan Doberstein) #9

the processing of your messages - how you work with your messages and what you do with the messages.


(Ben van Staveren) #10

If you route messages to streams based on “stream rules” (as set up on the stream page), then you must run message processors before pipelines.

If you run pipelines before message processors, you must attach them to “All Messages” stream since nothing has been routed yet.


(system) closed #11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.