GELF TCP Input Data to specific index

I need to put all data ingested by a specific TCP Input entry into a specific index user made. For this in the UI of Graylog I created a Stream on this index. Checked “Remove matches from ‘All messages’ stream”.

When create/edit the TCP Input I cannot find any place for Stream binding.

I created a single stream rule “Rule always matches” for the stream created above.

How do I connect my specific TCP Input to this Stream (and thus index)?

I want other TCP Inputs to keep storing record’s on Graylog’s default index - but this one. I noticed that other TCP Inputs started ingest the record in my user-made index too.


Other SIEMs usually resolve as

sourcetype = …
index = my_index

with a direct connection of the TCP Input to the desired index.

By default, messages from all inputs will land in the All Messages stream.

You can easily sort messages into other streams (eg. other indicies) by simply placing Pipeline Rules on the All Messages stream.

Eg. rule to the effect of if input = x, then sent to stream y.

thank you @tellistone

I would like not to touch the Graylog’s All Messages Stream.

what if Instead:

I create my own stream connected to my index, and without any rule
Have the “Remove matches from ‘All messages’ stream” checked for this stream.
Create a new Pipeline.
Connect this pipeline to my stream.
Create a Pipeline rule.

rule "Input for Oracle Audit"
	  from_input("My TCP Input")

Add this pipeline rule to Stage 0 of my pipeline

I tried the above, but the message went to Stream “All Messages”, and not to my Stream.

If that is your pipeline then you may need route-stream configured

rule "Input for Oracle Audit"
	  from_input("My TCP Input")

added the ```

but message still goes to All Messages

Have you tried to use the “Simulator” to see what’s happing?

Did you check you logs? Maybe something in there might help.

I did. But what am I supposed to find there?
The Simulation results are identical with Original message.
May be I am not getting something, sorry, but my Graylog knowledge is limited

What did the results look like?

No problem were here to help.

What really does help is showing us what you see. In the form of logs , errors/warning and even Information. The completed configuration of what you are executing. I can only guess from over here on what’s happening in your environment without see what you see.

thank you @gsmith

what should I upload?

How about your full Pipeline as you have it now.
Also when you run the simulator can you show us the output.

What I aim is that what loads on TCP Input “Oracle Audit TCP Input” goes to index “Oracle Unified Audit”.

last image pipeline rule

Ok, for future reference those screen shots are really, really hard to read. I had to copy them and use another application to read it.

For testing purposes lets use a unique FIELD call DB_NAME as shown below.

Now create a this pipeline rule.

Rule "Route to stream"
    has_field("DB_NAME") AND contains(to_string($message.DB_NAME), "dblive01")
    route_to_stream(name: "Oracle Audit Stream", remove_from_default: true);

NOTE: If the stream name “Oracle Audit Stream” does not work you may need to us the Stream ID instead.

route_to_stream(id:"the_ID _of_the_stream");

To get the ID click on the stream you want and the ID would be located in your URL as shown below


make sure your configuration is set with the pipeline processor after message filter chain
Something like this.

To find your Message Processors Configuration Navigate to System/Configurations.
I haven’t used from_input in my rules yet but I wanted to see if you pipeline is able to route to a stream first, I think this is the easiest way testing your pipeline out . Then we can go from there.

EDIT: You may need to wait a few minutes to see results, this depends on your environment.

I tried with both stream name and id. in both cases the message went to All Messages.

|3|Pipeline Processor|active|
|4|Message Filter Chain|active|

Should be this way

|3|Message Filter Chain|active|
|4|Pipeline Processor|active|

They are in default order, as per Graylog install. I have not changed anything.

You need this order.

Click the “Update” button and adjust it.