# Pipeline Not Getting Throughput

**URL:** <https://community.graylog.org/t/pipeline-not-getting-throughput/30512>\
**Category:** Graylog Central (peer support)\
**Created:** [October 31, 2023, 4:20pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512 "2023-10-31T16:20:45Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![poisedforflight](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/poisedforflight/32/6729_2.png) [@poisedforflight](https://community.graylog.org/u/poisedforflight)\
**Post date:** [October 31, 2023, 4:20pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/1 "2023-10-31T16:20:46Z")

</div>

I am trying to implement the pipeline rules found here but it’s not processing any messages: [Sonicwall Pipeline Rules - Templates and Rules Exchange / Pipeline Rules - Graylog Community](https://community.graylog.org/t/sonicwall-pipeline-rules/27587)

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/9/90eae372910547d6ac93f43a206669a80f1955d7.png)

```auto
rule "Extract: Sonicwall Extraction"
when
    has_field("source") AND contains(to_string($message.source), "192.168.1.1", true)
then
    set_fields(
        fields:key_value(
            value:to_string($message.message),
            //remove double quotes from keys and values
            trim_value_chars:"\"", 
            trim_key_chars:"\""
            )
        );
end

```

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/6/699cbf753200c80cb814329b9b075fdec4583032.png)

This is on GL 5.0.

Anyone have any suggestions or is there any other info I can supply that would be helpful?

---

<div class="post-metadata">

**Author:** ![faen](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@faen](https://community.graylog.org/u/faen)\
**Post date:** [October 31, 2023, 4:32pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/2 "2023-10-31T16:32:02Z")

</div>

Is the pipeline connected to the “Ridgeland SonicWall” stream?

In my application, I have my sonicwall only going into it’s own stream and not the Default Stream - not sure if that will make any difference or not, but might be worth a try if the pipeline is connected to the correct stream. Let me know if you want a screenshot of the pipeline connection, but it’s just below the “details” section. This is at the pipeline level and not the stage/rule level.

---

<div class="post-metadata">

**Author:** ![poisedforflight](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/poisedforflight/32/6729_2.png) [@poisedforflight](https://community.graylog.org/u/poisedforflight)\
**Post date:** [October 31, 2023, 4:33pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/3 "2023-10-31T16:33:42Z")

</div>

It is:

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/22c285de545dc18d7485d218dcd4783309541938.png)

---

<div class="post-metadata">

**Author:** ![faen](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@faen](https://community.graylog.org/u/faen)\
**Post date:** [October 31, 2023, 4:44pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/4 "2023-10-31T16:44:44Z")

</div>

Thanks for the quick confirmation.

As next steps, I would try:

1. See if you can take the messages out of the default stream, so they are only in the sonicwall stream, and if that changes the behavior.
2. Ensure that you are looking at the most recent messages in search, and not an older message that may not have been processed (I know, obvious, but this was a stumbling block for me once, so I am throwing it out there)
3. Modify the rule to just write something for every message, so no “when” clause, just a set\_field processed=true or something like that to try to isolate the error. Might be easier to make a new rule and move the rule in question to a later stage.

Hope that helps!

---

<div class="post-metadata">

**Author:** ![poisedforflight](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/poisedforflight/32/6729_2.png) [@poisedforflight](https://community.graylog.org/u/poisedforflight)\
**Post date:** [October 31, 2023, 4:53pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/5 "2023-10-31T16:53:36Z")

</div>

> [@faen](#):
>
> set\_field processed=true

1. I set to remove from default stream, no change
2. definitely looking at new messages
3. could you help me out with this. I’m not well versed in pipelines at all, trying to learn. What would the rule look like?

---

<div class="post-metadata">

**Author:** ![faen](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@faen](https://community.graylog.org/u/faen)\
**Post date:** [October 31, 2023, 5:06pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/6 "2023-10-31T17:06:24Z")

</div>

Here’s a very simple rule that should result in a single additional field called “Processed”

```auto
rule "Test Rule"
When
    has_field("message")
Then
    set_field("Processed","true");
End

```

If you aren’t seeing those, I would focus on the input/stream area. Have you checked the graylog application log?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [November 14, 2023, 5:07pm UTC](https://community.graylog.org/t/pipeline-not-getting-throughput/30512/7 "2023-11-14T17:07:17Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.