Messaging pipeline not changing the source field


#1

Hello,
I’m using Graylog 2.4.3 and having an issue with my first go at messaging pipelines. I have messages (a lot) that are spread across multiple extractors and most of the fields are getting pulled in correctly. The issue is the ‘source’ field (it’s incorrectly extracting the first portion of the incorrectly formatted ‘syslog’ messages to the current year). So the ‘source’ for a lot of devices is just ‘2018.’

What I would like to do is not mess with updating all of the extractors to try and pull in the source correctly but use a messaging pipeline to just change the ‘source’ field to the value of the ‘hostname’ that gets extracted from the message. Example raw message:

authmgr[4107]: <132094> <4107> MIC failed in WPA2 Key Message 2 from Station 24:0d:c2:8d:cd:f1 ac:a3:1e:87:c2:01 HIL-AP-55

and this is what my pipeline rule looks like (it runs on the ‘all messages’ stream):

rule “change_aruba_hostname"
when
to_bool(regex(”\\d\\d\\d\\d",to_string($message.source)).matches)
then
set_field(“source”,to_string($message.HOSTNAME));
end

HOSTNAME is the field that is getting populated by the extractor but this has not been applying to any messages where I would expect it to apply.

I’ve also tried without the ‘to_bool()’ function and tried something like this to no avail:

rule “change_aruba_hostname"
when
regex(”\\d\\d\\d\\d",to_string($message.source)).matches == true
then
set_field(“source”,to_string($message.HOSTNAME));
end

Any help is greatly appreciated!

EDIT: added correct amount of ‘’ slashes in my sample rule since this forum platform took one away as an escape character.


Wrong Source ID for Aruba IAP 215 Logs
(Jochen) #2

What’s the message processor order in your Graylog cluster? See System/Configurations.

You can properly escape your text snippets, see http://commonmark.org/help/.

Example:

```
Some text
More text
````

#3

Thanks for your reply!

Here is my order:

1 Message Filter Chain
2 GeoIP Resolver
3 AWS Instance Name Lookup
4 Pipeline Processor

Didn’t know to check for this but it looks like it should process correctly given this order.


(system) #4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.