[Graylog Coommunity 4.2.5] Need help with a regex pipeline

Graylog Central (peer support)
,
1

1. Describe your incident:
I have the following log messages from HPE network switches.

*<190>Jan 19 17:07:22 2022 pr1net161 %%10SSHS/6/SSHS_LOG: User networkad logged out from 10.4.28.27 port 41078*

As you can see, it add a year after the date, so Graylog always parse the source as 2022 :). So I want to pick up the actual source, in this case pr1net161

I have created a simple pipeline rule, but it keep return an empty set for some reason. It changes the source from 2022 to {}. Yes, to 2 curly brackets.

2. Describe your environment:

  • OS Information:
  • Linux RedHar 8.1 on baremetal
  • Package Version:
  • Graylog 4.2.5 fresh install with deployment for production intention

3. What steps have you already taken to try and solve the problem?

  • I have validated my regex
  • I have started a debug and found out I keep getting curly brackets

4. How can the community help?

  • I would appreciate it if someone can help me to find out what is wrong with my pipeline rule.

Thank you very much.

Sam

2

It would help if you posted the rule and regex you are using…

3

Apologies for not posting my solutions. Here is the simple pipeline rule, which is applied to the stream, just return invalid source.

rule "FlexFabric hostname resolver"
when
    has_field("source") AND has_field("message") 
then
    let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string($message.message));
    set_field("source", to_string(realSwitchHostName));
end

Thank you,
Sam

4

It looks ok to me… maybe you don’t need the flag modifiers in your regex? (?i) If you want case insensitive, just enforce lowercase on the value string:

let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string(lowercase($message.message)));

you can also use the debug function to help find out what is going on in the function…

5

Thank you for your input. I have tried with no flag modifier and also with only lowercase , still no luck. I was not able to find the debug logs when I added it to the rule. I looked in my logs directory and was not able to find it.

6

anything using the debug() function shows up in Graylog logs:

tail -f /var/log/graylog-server/server.log

rule "FlexFabric hostname resolver"
when
    has_field("source") AND has_field("message") 
then
    let realSwitchHostName = regex("(?i)[ps][rt]1net[0-9]*", to_string(lowercase($message.message)));
    set_field("source", to_string(realSwitchHostName));
   debug(concat("======= message: ", $message.message);
   debug(concat("======= realSwitchHostName : ", realSwitchHostName );
end
7

Thank you. I was able to find the logs. I was looking in the var/log/messages before .

Here we go.

2022-01-20T12:53:32.011-06:00 INFO [Function] PIPELINE DEBUG: ======= message: 2022 pr1net161 %%10SSHS/6/SSHS_AUTH_TIMEOUT: Authentication timed out for 10.10.10.10

2022-01-20T12:53:32.011-06:00 INFO [Function] PIPELINE DEBUG: ======= realSwitchHostName: {}

You can see it keep saying the name is {}

Thanks,
Hosam

8

Hmm - technically it is the first match you want so you can try:

set_field("source", realSwitchHostName["0"]);

My thinking it the brackets you are seeing are the set of matches. I don’t think this will work since it seems like for some reason regex finds nothing, on the other hand it is more precise than what is failing, and it is always good to be precise… and I can think of nothing else at the moment.

9

Thank you very much for trying to help me out. I did initially try calling the first returned arguments with no luck. I have also validated and manually tested my regex in simple java code and using online regex validators, and it seems to work well.

10

I can’t see why it won’t work - frustrating - what does your message Processor Configuration look like? On mine the pipeline processor (3) is after the Message Filter Chain (2)

The only other thing I can think of is convert it to GROK, but that is just a wrapper on regex…

grrr…

11

Here is my order, and it is working fine for all other pipelines I have.

1 Pipeline Processor active
2 Message Filter Chain active
3 AWS Instance Name Lookup disabled
4 GeoIP Resolver Disabled

Thanks for your help. I’ll keep trying.

-Sam

12

Actually I have the filter chain first then the pipeline. I copy/pasted to you when I was testing. I have all other pipelines working well in the following order, except this one I’m asking help with.

1 Message Filter Chain active
2 Pipeline Processor active
3 AWS Instance Name Lookup disabled
4 GeoIP Resolver disabled

-Sam

13

While I would rather figure out what problem Graylog/Elastic has with a regex that proves out to work everywhere else… I tested turning it into a grok and had positive results.

Created grok pattern (System->Grok Patterns menu):
SNAME [ps][rt]1net[0-9]*

Then had these lines in the rule:

let realSwitchHostName = grok("%{SNAME:source}", lowercase(to_string($message.message)),true);
set_fields(realSwitchHostName);
14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.