# Troubles writing a pipeline for Firepower Management Center logs

**URL:** <https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948>\
**Category:** Graylog Central (peer support)\
**Created:** [April 12, 2019, 11:10am UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948 "2019-04-12T11:10:55Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mariusgeonea](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/mariusgeonea/32/2162_2.png) [@mariusgeonea](https://community.graylog.org/u/mariusgeonea)\
**Post date:** [April 12, 2019, 11:10am UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/1 "2019-04-12T11:10:55Z")

</div>

Dear all,  
i have tons of messages coming from a Firepower Management Center that i need to parse. here is one example

```auto
<113>Apr 12 09:34:04 aus-fmc01 firepoweralert: [133:27:2] dcerpc2: Connection-oriented DCE/RPC - Invalid major version [Impact: Potentially Vulnerable] From "AUS-IPS-FPR-01" at Fri Apr 12 09:29:37 2019 UTC [Classification: Potentially Bad Traffic] [Priority: 2] {tcp} 10.163.52.36:53624 (unknown)->10.163.50.116:49156 (unknown)

```

I’m able to successfully do it with a grok extractor that looks like this

```auto
%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)

```

the problem with the grok extractors eat a lot of CPU which causes the load on it to go close to 100%

Now i’m trying to write a pipeline for parsing these messages but i’m having issues with the rule creation, it gives plenty of errors specially for characters like [ , ", } and so on…

my current rule that i’m trying to write looks like this

```auto
rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok(pattern: "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)", value: message_field, only_named_captures: true);
	
	set_fields(fmc0);

end

```

does anyone know how i can write the rule in such a way that i’d be able to parse the message and have no errors?

Thanks,  
Marius.

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [April 12, 2019, 8:38pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/2 "2019-04-12T20:38:53Z")

</div>

Add `^` to the beginning of your Grok pattern and watch the load go down 😉

---

<div class="post-metadata">

**Author:** ![mariusgeonea](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/mariusgeonea/32/2162_2.png) [@mariusgeonea](https://community.graylog.org/u/mariusgeonea)\
**Post date:** [April 12, 2019, 9:53pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/3 "2019-04-12T21:53:31Z")

</div>

Hi Ben,

i have added it but still gives plenty of errors and i can’t save it… now it looks like this

```auto
rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok(pattern: "^%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)", value: message_field, only_named_captures: true);
	
	set_fields(fmc0);

end

```

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [April 13, 2019, 7:59am UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/4 "2019-04-13T07:59:57Z")

</div>

You can’t save it? o\_O That’s weird, usually means there’s a typo in the code somewhere. Adding the `^` to the grok pattern anchors it to the beginning of the string, so that shouldn’t be it (it would also speed up regex parsing and bring load down).

Personally I would take the entire grok pattern and visit the System \> Grok Patterns page to save it as a single named pattern, and then `grok("%{MYGROKPATTERN}", to_string($message.message), true)` but that’s just my personal preference.

---

<div class="post-metadata">

**Author:** ![mariusgeonea](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/mariusgeonea/32/2162_2.png) [@mariusgeonea](https://community.graylog.org/u/mariusgeonea)\
**Post date:** [April 13, 2019, 12:38pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/5 "2019-04-13T12:38:23Z")

</div>

thanks for the idea, i’ve saved that grok pattern, and now when i’ve wrote my rule didn’t gave any errors and managed to save it.

but no message is getting parsed…

i have the felling that the way i’ve wrote the rule is wrong

```auto
rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok("%{MYGROKPATTERN}", to_string($message.message), true);
	
	set_fields(fmc0);

end

```

can you tell me what’s wrong with it? or maybe can you gimme the correct solution on how i should write this rule, please?

---

<div class="post-metadata">

**Author:** ![mariusgeonea](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/mariusgeonea/32/2162_2.png) [@mariusgeonea](https://community.graylog.org/u/mariusgeonea)\
**Post date:** [April 13, 2019, 12:48pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/6 "2019-04-13T12:48:49Z")

</div>

Ben, i must apologize, the rule works perfect, my problem was that i was missing a character in the graylog rule, the one which i’ve posted here was perfect.

Many many thanks for the help with this one.

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [April 13, 2019, 2:26pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/7 "2019-04-13T14:26:35Z")

</div>

😃 no worries, a typo is easily made. Glad to see it’s working! 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [April 27, 2019, 2:26pm UTC](https://community.graylog.org/t/troubles-writing-a-pipeline-for-firepower-management-center-logs/9948/8 "2019-04-27T14:26:37Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.