Troubles writing a pipeline for Firepower Management Center logs

Graylog Central (peer support)
1

Dear all,
i have tons of messages coming from a Firepower Management Center that i need to parse. here is one example

<113>Apr 12 09:34:04 aus-fmc01 firepoweralert: [133:27:2] dcerpc2: Connection-oriented DCE/RPC - Invalid major version [Impact: Potentially Vulnerable] From "AUS-IPS-FPR-01" at Fri Apr 12 09:29:37 2019 UTC [Classification: Potentially Bad Traffic] [Priority: 2] {tcp} 10.163.52.36:53624 (unknown)->10.163.50.116:49156 (unknown)

I’m able to successfully do it with a grok extractor that looks like this

%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)

the problem with the grok extractors eat a lot of CPU which causes the load on it to go close to 100%

Now i’m trying to write a pipeline for parsing these messages but i’m having issues with the rule creation, it gives plenty of errors specially for characters like [ , ", } and so on…

my current rule that i’m trying to write looks like this

rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok(pattern: "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)", value: message_field, only_named_captures: true);
	
	set_fields(fmc0);

end

does anyone know how i can write the rule in such a way that i’d be able to parse the message and have no errors?

Thanks,
Marius.

2

Add ^ to the beginning of your Grok pattern and watch the load go down :wink:

3

Hi Ben,

i have added it but still gives plenty of errors and i can’t save it… now it looks like this

rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok(pattern: "^%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:tag}: \[%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}:%{NUMBER:UNWANTED}\] %{WORD:app_protocol}: %{DATA:message_firepower} \[Impact: %{FIREPOWER_IMPACT:impact}\] From \"%{DATA:sensor_IP_or_name}\" at %{DAY:UNWANTED} %{MONTH:UNWANTED} %{MONTHDAY:UNWANTED} %{TIME:UNWANTED} %{YEAR:UNWANTED} %{WORD:UNWANTED} \[Classification: %{FIREPOWER_CLASSIFICATION:classification}\] \[Priority: %{NUMBER:priority}\] \{%{FIREPOWER_PROTOCOL:transport_prot}\} %{IPV4:src_IP}:%{NUMBER:src_port} \(%{FIREPOWER_COUNTRY:src_country}\)->%{IPV4:dst_IP}:%{NUMBER:dst_port} \(%{FIREPOWER_COUNTRY:dst_country}\)", value: message_field, only_named_captures: true);
	
	set_fields(fmc0);

end
4

You can’t save it? o_O That’s weird, usually means there’s a typo in the code somewhere. Adding the ^ to the grok pattern anchors it to the beginning of the string, so that shouldn’t be it (it would also speed up regex parsing and bring load down).

Personally I would take the entire grok pattern and visit the System > Grok Patterns page to save it as a single named pattern, and then grok("%{MYGROKPATTERN}", to_string($message.message), true) but that’s just my personal preference.

5

thanks for the idea, i’ve saved that grok pattern, and now when i’ve wrote my rule didn’t gave any errors and managed to save it.

but no message is getting parsed…

i have the felling that the way i’ve wrote the rule is wrong

rule "FMC rule 2"
when
    has_field("message")
then

	let message_field = to_string($message.message);

	let fmc0 = grok("%{MYGROKPATTERN}", to_string($message.message), true);
	
	set_fields(fmc0);

end

can you tell me what’s wrong with it? or maybe can you gimme the correct solution on how i should write this rule, please?

6

Ben, i must apologize, the rule works perfect, my problem was that i was missing a character in the graylog rule, the one which i’ve posted here was perfect.

Many many thanks for the help with this one.

7

:smiley: no worries, a typo is easily made. Glad to see it’s working! :slight_smile:

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.