Help me get value out of syslogs

New to Graylog Community? READ-ME FIRST Guides
1

Hi folks,
I have an in-house openwrt target that generates syslogs.The messages are recieved by syslog-ng which stores them to file and sends them further to graylog. It works but many of the possible fields are not extracted.For example I have syslog-entries looking like this

Oct 10 07:59:11 10.10.11.139 1 2025-08-07T23:18:15.596646+00:00 prx321-mydev8-eth /usr/sbin/swm-proxy-be 14988 - [meta sequenceId="26644"] APP_NAME:swm-proxy-be[T 43],APP_VERSION:1.0,MODULE_NAME:mmx/events/dispatcher.cpp(171),Formed RawEvent for "" with data: "{"name":"Device._mecAccess.X_mydev_COM_BlockedCPEs.CPE.{i}.","field":"LastChanged","keys":"84:93:0c:01:13:b0","value":"2025-08-07T23:18:15+00:00"}

Oct 10 07:59:11 10.10.11.139 1 2025-08-07T23:18:15.596646+00:00 prx321-mydev8-eth sysrepo-plugind 21045 - [meta sequenceId="26645"] APP_NAME:mmx-sysrepo[T 39],APP_VERSION:2.2.150,MODULE_NAME:mmxnc-plugin-2.0/plugin.c(883),Start updating RUNNING datastore.

Oct 10 07:59:11 10.10.11.139 1 2025-08-07T23:18:15.596646+00:00 prx321-mydev8-eth /usr/sbin/mmx-ep 7388 - [meta sequenceId="26646"] APP_NAME:mmx-ep[T 41],APP_VERSION:1.2.0,MODULE_NAME:ep_worker.c(14830),Got event task. Working on it

Oct 10 07:59:12 10.10.11.139 1 2025-08-07T23:18:15.914188+00:00 prx321-mydev8-eth kernel - - [meta sequenceId="26798"] [ 3307.949460] UBIFS (ubi1:0): un-mount UBI device 1

Oct 10 07:59:12 10.10.11.139 1 2025-08-07T23:18:15.914188+00:00 prx321-mydev8-eth kernel - - [meta sequenceId="26799"] [ 3307.953017] UBIFS (ubi1:0): background thread "ubifs_bgt1_0" stops

I would like fields like APP_NAME, APP_VERSION, MODULE_NAME to show up as fields and be able to filter on.
I have read that there are extractors, pipelines and a GROK language
I also found that syslog-ng could help my with functions like graylog2 and also syslog

Since I’m a newbie I just want someone with experience to point me in the right direction to get most value from my syslogs with minimum effort and best results.
When just sending the messages to an syslog-udp input there is just a huge message field containing it all.

2

Hey @aka_se,

This can be achieved with pipelines and rules, specifically you would want to look at either creating a grok pattern to parse out your syslog into fields or more bluntly extracting the desired content into fields using regex.

Below is some slop GPT spewed out which works.

rule "extract app fields (grok)"
when
  has_field("message") &&
  contains(to_string($message.message), "APP_NAME:")
then
  let m = grok(
    pattern: "APP_NAME:%{DATA:APP_NAME},APP_VERSION:%{DATA:APP_VERSION},MODULE_NAME:%{DATA:MODULE_NAME},",
    value: to_string($message.message)
  );
  set_fields(m);    // creates fields APP_NAME, APP_VERSION, MODULE_NAME
end
3

Great,
That will get me started with the messages as raw-udp
I tried take the syslog line
<34>1 2025-01-03T14:07:15.003Z ``mymachine.example.com`` su 12345 ID47 - 'su root' failed for user on /dev/pts/0
from this page for RFC5424

but even in that case the syslog input in Graylog was unable to extract the fields which are mentioned right below that line like App-name, Process Id, Hostname etc
I guess I’m doing something wrogn as a beginner…