Route_to_stream in pipeline processor stops processing of messages

Graylog Central (peer support)
,
1

Hi all,

I need some help with the route_to_stream function of the pipeline processor.

I am currently playing around with graylog and the pipeline processor to enrich the log messages. The first is that I want to route message of certain applications into separate streams. For this I created a new stream and created the following rule:

rule "route test"
when
  has_field("application") && to_string($message.application) == "test"
then
  route_to_stream(id: "58cc0a2e2bf0923e5c4b4acc");
end

If I connect the pipeline using this rule with the main stream graylog stops processing incoming messages. The process buffer starts growing and if it is full the disk journal… No error message in the logs and the trace messages I activated for the pipeline plugin looks fine for me, as well. Probably I don’t see it. Using the streams name had the same result and the stream is in running state.

I get the stream id on the streams “details page” from the browser url …/streams/58cc0a2e2bf0923e5c4b4acc/search. Would not know about a better place or another thing looking like the ID.

The graylog version is the current one. Version 2.2.2

It would be great if someone could help me.

Thomas

2 Likes
2

Hi Thomas,

Try this:

rule “route test”
when
has_field(“application”) && contains(to_string($message.application),“test”)
then
route_to_stream(id: “58cc0a2e2bf0923e5c4b4acc”);
end

3

The condition works. I checked that with setting a field with some data.

It is definitely the route_to_stream() function which breaks the processing.

4

I found this warning in the logs could it be related to my problem?

2017-03-20_16:45:38.22349 2017-03-20 16:45:38,223 WARN : org.graylog2.shared.events.DeadEventLoggingListener -  Received unhandled event of type <org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.State> from event bus <AsyncEventBus{graylog-eventbus}>

Edit: Yes it is releated. As soon I save the rule with the routing function I can see this message in the logs and graylog stops processing messages completely. Means from this time on they don’t leave the process buffer and don’t get saved to ES. Only a restart of graylog helps.

5

@staelche can you please fill a github issue for this:

thank you

6

@Jan: Yes, I will. In the meantime i figured out that the routing is not the problem but I conntected already other rules to the stream. One quite simple rule triggers the mentioned warning in the logs and the process buffer fills up.Still a bug ticket worth I would guess.

But let me reproduce this first with the latest VM v2.2.2 installed so that you can reproduce it as well, if needed. Currently I use your appliance but updated on my own from 2.2.1 to 2.2.2 (should not be the problem but could be).

1 Like
7

Any luck solving this issue?
Experiencing the same thing with my lab using VM v2.2.1

Thanks,
Jonathan

8

No sorry.

I’m currently off work.

Maybe you want wo create a bug report and link it here?