# GELF TCP Input Data to specific index

**URL:** <https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810>\
**Category:** Graylog Tech Challenges\
**Created:** [November 20, 2021, 9:38pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810 "2021-11-20T21:38:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 20, 2021, 9:38pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/1 "2021-11-20T21:38:51Z")

</div>

I need to put all data ingested by a specific TCP Input entry into a specific index user made. For this in the UI of Graylog I created a Stream on this index. Checked “Remove matches from ‘All messages’ stream”.

When create/edit the TCP Input I cannot find any place for Stream binding.

I created a single stream rule “Rule always matches” for the stream created above.

Question:  
How do I connect my specific TCP Input to this Stream (and thus index)?

I want other TCP Inputs to keep storing record’s on Graylog’s default index - but this one. I noticed that other TCP Inputs started ingest the record in my user-made index too.

regards  
Altin

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 20, 2021, 11:49pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/2 "2021-11-20T23:49:48Z")

</div>

Other SIEMs usually resolve as

[default]  
[tcp://50001]  
sourcetype = …  
index = my\_index

with a direct connection of the TCP Input to the desired index.

---

<div class="post-metadata">

**Author:** ![tellistone](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tellistone/32/10200_2.png) [@tellistone](https://community.graylog.org/u/tellistone)\
**Post date:** [November 23, 2021, 12:45pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/3 "2021-11-23T12:45:00Z")

</div>

By default, messages from all inputs will land in the All Messages stream.

You can easily sort messages into other streams (eg. other indicies) by simply placing Pipeline Rules on the All Messages stream.

Eg. rule to the effect of if input = x, then sent to stream y.

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 23, 2021, 11:52pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/4 "2021-11-23T23:52:53Z")

</div>

thank you @tellistone

I would like not to touch the Graylog’s All Messages Stream.

what if Instead:

I create my own stream connected to my index, and without any rule  
Have the “Remove matches from ‘All messages’ stream” checked for this stream.  
Create a new Pipeline.  
Connect this pipeline to my stream.  
Create a Pipeline rule.

```auto
rule "Input for Oracle Audit"
	when
	  from_input("My TCP Input")
	then
	end

```

Add this pipeline rule to Stage 0 of my pipeline

I tried the above, but the message went to Stream “All Messages”, and not to my Stream.

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 23, 2021, 11:56pm UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/5 "2021-11-23T23:56:43Z")

</div>

> [@altink](#):
>
> ````auto
> rule "Input for Oracle Audit"
> when
> from_input("My TCP Input")
> then
> end```
> 
> ````

If that is your pipeline then you may need route-stream configured

```auto
rule "Input for Oracle Audit"
	when
	  from_input("My TCP Input")
then
    route_to_stream(id:"5d8acba383d72e04cba96317");
end
```

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:19am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/6 "2021-11-24T00:19:12Z")

</div>

added the ```  
route\_to\_stream

```auto
but message still goes to All Messages
```

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 12:25am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/7 "2021-11-24T00:25:48Z")

</div>

Have you tried to use the “Simulator” to see what’s happing?

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/3/3ea8401ac274590674ad14917f2b99489f819e4d.png)

Did you check you logs? Maybe something in there might help.

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:34am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/8 "2021-11-24T00:34:05Z")

</div>

I did. But what am I supposed to find there?  
The Simulation results are identical with Original message.  
May be I am not getting something, sorry, but my Graylog knowledge is limited

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 12:37am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/9 "2021-11-24T00:37:57Z")

</div>

> [@altink](#):
>
> The Simulation results are identical with Original message.

What did the results look like?

> [@altink](#):
>
> sorry, but my Graylog knowledge is limited

No problem were here to help.

What really does help is showing us what you see. In the form of logs , errors/warning and even Information. The completed configuration of what you are executing. I can only guess from over here on what’s happening in your environment without see what you see.

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:42am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/10 "2021-11-24T00:42:22Z")

</div>

thank you @gsmith

what should I upload?

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 12:44am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/11 "2021-11-24T00:44:17Z")

</div>

How about your full Pipeline as you have it now.  
Also when you run the simulator can you show us the output.

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:48am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/12 "2021-11-24T00:48:25Z")

</div>

![Screenshot 2021-11-24 at 01-46-21 Graylog Web Interface - Simulate processing](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/1/14143d03571e70ca713c0d0912c4a92eac35e345.png)

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:49am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/13 "2021-11-24T00:49:32Z")

</div>

![Screenshot 2021-11-24 at 01-48-44 Graylog - Pipeline Pipeline 01](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/265e912452f70b3ac4591a91ad2d3783a599ea94.png)

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:52am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/14 "2021-11-24T00:52:10Z")

</div>

![Screenshot 2021-11-24 at 01-50-07 Graylog Web Interface - Inputs](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/6/610a325701f0641e871c2a9eb4dba4c53494c8af.png)  
 ![Screenshot 2021-11-24 at 01-51-24 Graylog - Streams](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/5/58fc86ac0ae48ee4e7d540bd25674a0f7ce76b0e.png)

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 12:57am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/15 "2021-11-24T00:57:07Z")

</div>

What I aim is that what loads on TCP Input “Oracle Audit TCP Input” goes to index “Oracle Unified Audit”.

last image pipeline rule

 ![Screenshot 2021-11-24 at 01-54-28 Graylog - Pipeline rule P-Rule Input for Oracle Audit](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/a/a4fc8ed1f766943842df8a415909642a237dbac2.png)

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 1:10am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/16 "2021-11-24T01:10:22Z")

</div>

Ok, for future reference those screen shots are really, really hard to read. I had to copy them and use another application to read it.

For testing purposes lets use a unique **FIELD** call **DB\_NAME** as shown below.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/7/73a3ce50aa3e1d042ec006ab03ecfcdb1c9720f7.png)  
Now create a this pipeline rule.

```auto
Rule "Route to stream"
when
    has_field("DB_NAME") AND contains(to_string($message.DB_NAME), "dblive01")
then
    route_to_stream(name: "Oracle Audit Stream", remove_from_default: true);
end

```

NOTE: If the stream name “Oracle Audit Stream” does not work you may need to us the **Stream ID** instead.

`route_to_stream(id:"the_ID _of_the_stream");`

To get the ID click on the stream you want and the ID would be located in your URL as shown below

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/5/54cd579ea6f4f9c0fd272860be90d41e157f5156.png)

NEXT  
make sure your configuration is set with the **pipeline processor** after **message filter chain** …  
Something like this.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/9/92f385058301303385f49e7af0a1e0f67ce9f2bc.png)

To find your Message Processors Configuration Navigate to System/Configurations.  
I haven’t used **from\_input** in my rules yet but I wanted to see if you pipeline is able to route to a stream first, I think this is the easiest way testing your pipeline out . Then we can go from there.

**EDIT** : You may need to wait a few minutes to see results, this depends on your environment.

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 1:20am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/17 "2021-11-24T01:20:45Z")

</div>

I tried with both stream name and id. in both cases the message went to All Messages.

```auto
|3|Pipeline Processor|active|
|4|Message Filter Chain|active|

```

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 1:22am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/18 "2021-11-24T01:22:34Z")

</div>

> [@altink](#):
>
> ```auto
> |3|Pipeline Processor|active|
> |4|Message Filter Chain|active|
> 
> ```

Should be this way

```auto
|3|Message Filter Chain|active|
|4|Pipeline Processor|active|

```

---

<div class="post-metadata">

**Author:** ![altink](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/altink/32/3446_2.png) [@altink](https://community.graylog.org/u/altink)\
**Post date:** [November 24, 2021, 1:30am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/19 "2021-11-24T01:30:26Z")

</div>

They are in default order, as per Graylog install. I have not changed anything.

 ![Screenshot 2021-11-24 at 02-28-46 Graylog - Configurations](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/8/8f4fa32b04b77b7901680d5e288e912932e2f328.png)

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [November 24, 2021, 1:31am UTC](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810/20 "2021-11-24T01:31:27Z")

</div>

You need this order.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/9/92f385058301303385f49e7af0a1e0f67ce9f2bc.png)  
Click the “Update” button and adjust it.

[Next page](https://community.graylog.org/t/gelf-tcp-input-data-to-specific-index/21810.md?page=2)