Pipeline is not working for other streams

Graylog Central (peer support)
,
1

Hi All

I have created a Pipeline named Test1 and connected with a newly created stream Logging user activity but it’s showing current throughput 0. Also when I connected the default All messages stream to pipeline Test1 then data start coming.

02%20PM
Screen Shot 2018-12-05 at 5.18.02 PM.png1901×395 45.3 KB
53%20PM
Screen Shot 2018-12-05 at 5.18.53 PM.png1886×377 43.9 KB

FYI - Logs is already coming in Logging user activity but in the pipeline, throughput is 0.

24%20PM
Screen Shot 2018-12-05 at 5.18.24 PM.png844×106 12.7 KB

It’s working for default All Messages stream but not working for newly created streams.

Here is the Message Processors Configuration order.

16%20PM
Screen Shot 2018-12-05 at 5.37.16 PM.png953×406 20.5 KB

Please help me with this.

Thanks

2

He @Tafsir_Alam

what is your processing order in system > configuration ? That might be the reason for that.

3

@jan Here is the processing order.

16%20PM
Screen Shot 2018-12-05 at 5.37.16 PM.png953×406 20.5 KB

4

Dear @Tafsir_Alam, I am also facing the same issue. Maybe we will discuss in this forum for further process.

5

The routing into streams is done in the “message filter chain” what in your case runs after the processing pipelines.

So if you change the processor configuration order that the processing pipeline runs after the message filter chain you can use the stream rules to route messages into streams and then run the processing pipelines on the data that is in the stream.

currently you would need a processing pipeline that routes the messages into the streams connected to all messages and then a second pipeline that does the extraction based on the data in the streams.

2 Likes
6

@jan Is this create any problem on the messages that coming on the graylog if I change the order that is processing pipeline runs after the message filter chain?

7

it depends on your processing and can’t be answered in general.

8

What Processing @jan ?

9

the processing of your messages - how you work with your messages and what you do with the messages.

10

If you route messages to streams based on “stream rules” (as set up on the stream page), then you must run message processors before pipelines.

If you run pipelines before message processors, you must attach them to “All Messages” stream since nothing has been routed yet.

1 Like
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.