I am missing functionaiilty in the input que. How to deal with that !?

[gsmith]

Hey @louis

I agree pipelines are not easy. There are tons of examples in this forum on HowTo’s , configurations, routing, re-naming, filtering messages, also use regex/Grok within the pipeline, etc…

@louis

I am not aware of a function which can convert extractors into a pipeline

Also in this forum. BUT if I understand this correct, the function of this extractor, then how to make a pipeline the same way, OR magically turn the Extractor into a pipeline?

Example #1

let’s say you have an Input for Linux devices, and you need to route a message that has a unique field “node_02” into another stream and send out an alert.

rule " alert"
when
  has_field("node_02")
then
  route_to_stream(id:"63094a92218139114d4923f2");
end

I tend to use stream ID’s instead of stream names.

Example #2

let’s say you have an Input for Linux devices, and you need to route a message that has a unique field “node_02” with specific data under that field call “Louis” into another stream and send out an alert.

rule “Route Node_02/Louis”

when
    has_field("node_02") AND contains(to_string($message.node_02, "Louis")
then
     route_to_stream(id:"63094a92218139114d4923f2");
end

Not only will you find example here in the forum but also in GitHub, there are Tag’s" here in the forum you can use for better search. Pipelines are so versatile I cant post all configurations here.

Dropping message.

Example#3

rule "discard Message with Louis"
    when
        has_field("node_02") AND  contains(to_string($message.node_02),  "Louis", true)
    then
        drop_message();
    end

@louis don’t take this the wrong way but these statements tell me you not really interested in Pipeline, probably because you don’t have enough knowledge.

@louis

• I am missing a picture showing about the graylog architecture. How function blocks like inputs, steams, extractors etc. are interconnected. If you know such a picture … please share it
• pipelines seems to be more powerfull than extractors but also not as easy to use
• I am not aware of a function which can convert extractors into a pipeline
• In point one 1) you suggest an check on a field as solution, that is what I do within the extractors, I was looking for something like “break” (I am done leave the pipeline / return from the pipeline function)
• I did read somewhere about a filter function, but I can not find such an function of “processing block” in the collection ^architecture^
• I did read somewhere that you can define the order in which function blocks are processed, no idea how to do that (or is that an enterprise feature perhaps)

To sum those up as follow:
Steps

• Pipelines

One of our Members took some time to demo this out for others, hope this helps.

• Graylog in a Nutshell (Diagram)

• Graylog Maintained Pipeline Examples

If you get stuck and need assistants, I’m sure someone here can help.

EDIT:
I forgot to add about Extractors, depending on what your trying to achieve you can create a REGEX extractor and attach a lookup table to it.
Here is one of mine, extractor type regular expression