# \[Graylog Coommunity 4.2.5\] Need help with a regex pipeline

**URL:** <https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406>\
**Category:** Graylog Central (peer support)\
**Tags:** pipeline-rules, regex-special-charac\
**Created:** [January 20, 2022, 12:25am UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406 "2022-01-20T00:25:16Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 20, 2022, 12:25am UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/1 "2022-01-20T00:25:16Z")

</div>

**1. Describe your incident:**  
I have the following log messages from HPE network switches.

`*<190>Jan 19 17:07:22 2022 pr1net161 %%10SSHS/6/SSHS_LOG: User networkad logged out from 10.4.28.27 port 41078*`

As you can see, it add a year after the date, so Graylog always parse the source as **2022** :). So I want to pick up the actual source, in this case **pr1net161**

I have created a simple pipeline rule, but it keep return an empty set for some reason. It changes the source from 2022 to {}. Yes, to 2 curly brackets.

**2. Describe your environment:**

- OS Information:

- Linux RedHar 8.1 on baremetal

- Package Version:

- Graylog 4.2.5 fresh install with deployment for production intention

**3. What steps have you already taken to try and solve the problem?**

- I have validated my regex
- I have started a debug and found out I keep getting curly brackets

**4. How can the community help?**

- I would appreciate it if someone can help me to find out what is wrong with my pipeline rule.

Thank you very much.

Sam

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 20, 2022, 3:50am UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/2 "2022-01-20T03:50:05Z")

</div>

It would help if you posted the rule and regex you are using…

> [@Tips for Posting Questions that Get Answers](https://community.graylog.org/t/tips-for-posting-questions-that-get-answers/21828):
>
> TIPS for Posting Questions that Get Answers: \*The following is a compendium of tips to help you organize your question and have better success for getting a solution to the incident you are having. First, a short list, then some detail on how to pull and post information from your systems. When you create a new Topic for the community to review with you: Search Graylog Documentation, the Graylog community forum, Google … the answer may be out there! Have a short informative subject such as:…

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 20, 2022, 2:13pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/3 "2022-01-20T14:13:45Z")

</div>

Apologies for not posting my solutions. Here is the simple pipeline rule, which is applied to the stream, just return invalid source.

```auto
rule "FlexFabric hostname resolver"
when
    has_field("source") AND has_field("message") 
then
    let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string($message.message));
    set_field("source", to_string(realSwitchHostName));
end

```

Thank you,  
Sam

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 20, 2022, 5:40pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/4 "2022-01-20T17:40:50Z")

</div>

It looks ok to me… maybe you don’t need the flag modifiers in your regex? `(?i)` If you want case insensitive, just enforce lowercase on the value string:

` let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string(lowercase($message.message)));`

you can also use the debug function to help find out what is going on in the function…

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 20, 2022, 5:59pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/5 "2022-01-20T17:59:59Z")

</div>

Thank you for your input. I have tried with no flag modifier and also with only lowercase , still no luck. I was not able to find the debug logs when I added it to the rule. I looked in my logs directory and was not able to find it.

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 20, 2022, 6:20pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/6 "2022-01-20T18:20:34Z")

</div>

anything using the `debug()` function shows up in Graylog logs:

`tail -f /var/log/graylog-server/server.log`

```auto
rule "FlexFabric hostname resolver"
when
    has_field("source") AND has_field("message") 
then
    let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string(lowercase($message.message)));
    set_field("source", to_string(realSwitchHostName));
   debug(concat("======= message: ", $message.message);
   debug(concat("======= realSwitchHostName : ", realSwitchHostName );
end

```

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 20, 2022, 6:56pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/7 "2022-01-20T18:56:10Z")

</div>

Thank you. I was able to find the logs. I was looking in the var/log/messages before .

Here we go.

2022-01-20T12:53:32.011-06:00 INFO [Function] PIPELINE DEBUG: ======= message: 2022 pr1net161 %%10SSHS/6/SSHS\_AUTH\_TIMEOUT: Authentication timed out for 10.10.10.10

2022-01-20T12:53:32.011-06:00 INFO [Function] PIPELINE DEBUG: ======= realSwitchHostName: {}

You can see it keep saying the name is **{}**

Thanks,  
Hosam

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 20, 2022, 9:33pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/8 "2022-01-20T21:33:00Z")

</div>

Hmm - technically it is the first match you want so you can try:

`set_field("source", realSwitchHostName["0"]);`

My thinking it the brackets you are seeing are the set of matches. I don’t think this will work since it seems like for some reason regex finds nothing, on the other hand it is more precise than what is failing, and it is always good to be precise… and I can think of nothing else at the moment.

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 21, 2022, 1:55pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/9 "2022-01-21T13:55:19Z")

</div>

Thank you very much for trying to help me out. I did initially try calling the first returned arguments with no luck. I have also validated and manually tested my regex in simple java code and using online regex validators, and it seems to work well.

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 21, 2022, 6:26pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/10 "2022-01-21T18:26:33Z")

</div>

I can’t see why it won’t work - frustrating - what does your message Processor Configuration look like? On mine the pipeline processor (3) is after the Message Filter Chain (2)

The only other thing I can think of is convert it to GROK, but that is just a wrapper on regex…

grrr…

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 21, 2022, 7:29pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/11 "2022-01-21T19:29:26Z")

</div>

Here is my order, and it is working fine for all other pipelines I have.

| 1 | Pipeline Processor | active |
| --- | --- | --- |
| 2 | Message Filter Chain | active |
| 3 | AWS Instance Name Lookup | disabled |
| 4 | GeoIP Resolver | Disabled |

Thanks for your help. I’ll keep trying.

-Sam

---

<div class="post-metadata">

**Author:** ![hmb104](https://avatars.discourse-cdn.com/v4/letter/h/87869e/32.png) [@hmb104](https://community.graylog.org/u/hmb104)\
**Post date:** [January 21, 2022, 7:32pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/12 "2022-01-21T19:32:19Z")

</div>

Actually I have the filter chain first then the pipeline. I copy/pasted to you when I was testing. I have all other pipelines working well in the following order, except this one I’m asking help with.

| 1 | Message Filter Chain | active |
| --- | --- | --- |
| 2 | Pipeline Processor | active |
| 3 | AWS Instance Name Lookup | disabled |
| 4 | GeoIP Resolver | disabled |

-Sam

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [January 21, 2022, 10:16pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/13 "2022-01-21T22:16:49Z")

</div>

While I would rather figure out what problem Graylog/Elastic has with a regex that proves out to work everywhere else… I tested turning it into a grok and had positive results.

Created grok pattern (System-\>Grok Patterns menu):  
`SNAME [ps][rt]1net[0-9]*`

Then had these lines in the rule:

```auto
let realSwitchHostName = grok("%{SNAME:source}", lowercase(to_string($message.message)),true);
set_fields(realSwitchHostName);

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [February 4, 2022, 10:17pm UTC](https://community.graylog.org/t/graylog-coommunity-4-2-5-need-help-with-a-regex-pipeline/22406/14 "2022-02-04T22:17:23Z")

</div>

Closed on February 4, 2022, 10:17pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.