# Doing a pipeline rule

**URL:** <https://community.graylog.org/t/doing-a-pipeline-rule/31062>\
**Category:** Graylog Central (peer support)\
**Created:** [December 28, 2023, 5:21am UTC](https://community.graylog.org/t/doing-a-pipeline-rule/31062 "2023-12-28T05:21:24Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![wilsonshow](https://avatars.discourse-cdn.com/v4/letter/w/df705f/32.png) [@wilsonshow](https://community.graylog.org/u/wilsonshow)\
**Post date:** [December 28, 2023, 5:21am UTC](https://community.graylog.org/t/doing-a-pipeline-rule/31062/1 "2023-12-28T05:21:24Z")

</div>

I am weak at doing a pipeline + regex rule. Would need help with the following.

I have this message coming in:

Dec 28 11:47:20 graylog01 filebeat[48973]: 2023-12-28T11:47:20.874+0800#011INFO#011[input.harvester]#011log/harvester.go:310#011Harvester started for paths: [/var/log/\*]#011{“input\_id”: “4bc7f922-ab75-4db4-bfe2-54f2f8acb5e7”, “source”: “/var/log/dmesg.1.gz”, “state\_id”: “native::594-64768”, “finished”: false, “os\_id”: “594-64768”, “old\_source”: “/var/log/dmesg.1.gz”, “old\_finished”: true, “old\_os\_id”: “594-64768”, “harvester\_id”: “ef0c3cee-a7b0-45e2-b185-c75976d750c2”}

But I would like to do a pipeline rule to make it nicer in looking, for example like the following format:

source:  
state\_id:  
finished:  
os\_id:  
old\_source:  
old\_finished:  
old\_os\_id:  
harvester\_id:

---

<div class="post-metadata">

**Author:** ![drewmiranda-gl](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/drewmiranda-gl/32/13376_2.png) [@drewmiranda-gl](https://community.graylog.org/u/drewmiranda-gl)\
**Post date:** [December 28, 2023, 5:14pm UTC](https://community.graylog.org/t/doing-a-pipeline-rule/31062/2 "2023-12-28T17:14:33Z")

</div>

Can you share how you are shipping these logs to graylog? Are you using filebeat? The message you provided appears to be a syslog message generated from filebeat.

The following blog post should be a good resource for getting started:

> **[Graylog Parsing Rules and AI Oh My!](https://graylog.org/post/graylog-parsing-rules-and-ai-oh-my/)**
>
> Graylog parsing rules can be tricky at times but in this blog you will find many ways to do this and also include the ability to use AI!

---

<div class="post-metadata">

**Author:** ![wilsonshow](https://avatars.discourse-cdn.com/v4/letter/w/df705f/32.png) [@wilsonshow](https://community.graylog.org/u/wilsonshow)\
**Post date:** [December 29, 2023, 1:38am UTC](https://community.graylog.org/t/doing-a-pipeline-rule/31062/3 "2023-12-29T01:38:34Z")

</div>

Hey,

I was trying different collector to send to different type of input.  
So for this example, I have configure the filebeat.yml and path it to /var/lib/\*.log. So it will send any file that will end with .log.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [January 12, 2024, 1:38am UTC](https://community.graylog.org/t/doing-a-pipeline-rule/31062/4 "2024-01-12T01:38:54Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.