JSON input and pipeline enrichment

Graylog Central (peer support)
1

The initial goal seemed trivial, use a pipeline to enrich my IDS streams with Threat Intel data. Unfortunately nothing I have tried has worked thus far.

First issue seems to be that if you select “remove matches from all messages” for a stream and then use that stream in a pipeline - the pipeline always registers 0 msgs/sec. In order to see messages in the pipeline, I have to leave the messages in All Messages and connect the pipeline to the All Messages stream instead.

The input(s) for this data are JSON feeds from Snort and Bro and I am using JSON extractors on the input to parse the data into useful fields.

However, no matter what I try (and I’ve following several google rabbitholes on this) none of my rule syntax seems to trigger - not even a simple has_field test rule. I’m thinking this is because the raw message is JSON, but I’m not sure.

This is happening on both Graylog 2.4.x and 3.x

2

he @chavez243ca sorry that you have such issues, from what you describe I can’t see a reason why a pipeline does not work if you connect that to a stream with messages. ONLY one single reason can be given for that:

The processing order in System / configuration

You should have the message filter chain before the processing pipeline

1 Like
3

Thanks - processing order was incorrect. I will see if that addresses the issues.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.