1. Describe your incident: Unable to split audit log messages into separate fields (by key-values) and prefixing these fields with “auditd_”. 2. Describe your environment: OS Information: Debian 10 LTS (4.19.0-21-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64 GNU/Linux) Package Version: Gr…

Set your Message Filter Chain to come before your pipeline processor … Message Filter Chain should have a lower number than the Pipeline Processor.

Check to see if if your Message Filter Chain is before your Pipeline Processor under Message Processor Configuration, IF NOT - this often causes no results in the pipeline. Also, you can watch in the logs to see what is going on using the debug() function - I added it below with a note that shows …

@tmacgbay :+1: I did a Copy & Paste in my lab with @foss4ever logs. Learned something new, Thanks [image]

Check to see if if your Message Filter Chain is before your Pipeline Processor under Message Processor Configuration Looks ok to me: [image] I have tried changing the “when” condition to just true on my pipeline rule as you suggested as well as replacing the rule code with what you posted, con…

For the sake of testing more thoroughly, I have also tried setting up a new pipeline and rule for a different stream “Debian server logs” with the same result of no log throughput whatsoever. So it seems this issue is not related to the audit log stream as such…

Sorry, I misread your initial explanation. But now, magically, everything seems to finally work :sweat_smile: Big thanks go out to yourself as well as @gsmith At this point I can’t help but wonder why the default Message Processors Configuration would be set in a way that seems to prevent pipelin…

I am not sure why it is set up that way… @dscryber do you have a contact with someone in engineering who could give the explanation of why the Message Filter Chain defaults to being after the Processor Pipeline (when all the cool kids like it to be first) and maybe why you would want it that way?

Thanks, @tmacgbay . Yes, I’ll bring this to engineering’s attention.

Sounds great. If that configuration could be changed it might save another Graylog newbie like myself many hours of frustration :slight_smile:

The default order is an unfortunate oversight. It is ordered alphabetically (with the full class name) For a long time we didn’t dare to change the default, because it will break the processing of existing setups. The next release will have the order fixed. (But only for new installations) See D…

[image] mpfz0r: Also of Note: The stream rules are become a separate message processor: Thank you so, so much!. I am really looking forward to that change. err, sorry to bother you, but is there a chance that the implicit processing within the syslog input would get separated in the same wa…

Pipeline rule to split log messages into key-value fields not working

Graylog Central (peer support)

nisow95612 August 30, 2022, 4:54pm 17

As I write in RAW Input with “Length-prefixed framing”, I can’t. Raw input works only for messages with newline or null-character framing. My messages have length-prefixed framing and if I send them to raw input they all join together into one big mess.

Topic		Replies	Views
Pipeline rule to extract key-value pair not working Graylog Central (peer support) pipeline-rules	19	2865	April 9, 2022
Syslog message not being parsed when using Graylog Sidecar with filebeat input Graylog Central (peer support) sidecar , filebeat-linux	28	2593	August 18, 2022
Alert processing query Graylog Central (peer support)	3	751	July 18, 2018
Issue with Key=Value Parser Pipeline for specific log-messages Graylog Central (peer support) pipeline-rules , debuggingpl	2	2101	April 27, 2021
Odd Pipeline/Stream Behavior Graylog Central (peer support)	9	567	January 5, 2024

Pipeline rule to split log messages into key-value fields not working

Related topics