Need help with extractor grok pattern

[Shaju (Devassy)]

Hi,

just installed graylog two day back, trying to figure out how all works need some help with my cisco firepower syslogs delivered as plain text to get it extracted for then adding it to the pipeline.

Sample:

May 05 2022 20:25:51 %FTD-1-400006: EventPriority: Low, DeviceUUID: cd5405c4-cea2-13eb-94ea-bdzc1md93886, InstanceID: 2, FirstPacketSecond: 2022-05-05T20:25:51Z, ConnectionID: 50568, AccessControlRuleAction: Allow, SrcIP: 178.189.7.12, DstIP: 192.16.1.31, SrcPort: 54022, DstPort: 443, Protocol: tcp, IngressInterface: CAG, EgressInterface: LAN, IngressZone: CAG, EgressZone: LAN, IngressVRF: Global, EgressVRF: Global, ACPolicy: Access Control Policy, AccessControlRuleName: 00000000-0000-0168-0700-c0eae4ec0b4a, Prefilter Policy: Default Prefilter Policy, Client: SSL client, ApplicationProtocol: HTTPS, WebApplication: Invalid ID, ConnectionDuration: 0, InitiatorPackets: 8, ResponderPackets: 8, InitiatorBytes: 443, ResponderBytes: 5584, NAPPolicy: Balanced Security and Connectivity, URLReputation: Unknown, URL: https://wawa.us.net

i am trying to extract these fields, few of them if pasted indivdually it gives me result but when joined it says null in grox debugger as well as when trying in the extractor. would be of great help if someone can point out what i am doing wrong. Thanks in advance.

%{SYSLOGTIMESTAMP:timestamp}, AccessControlRuleAction: %{WORD:ac_action}, AccessControlRuleName: %{DATA:ac_rule_name}, SrcIP: %{IP:src_addr}, DstIP: %{IP:dst_addr}, IngressZone: %{HOSTNAME:ingress_zone}, EgressZone: %{HOSTNAME:egress_zone}, ApplicationProtocol: %{WORD:proto}, URL: %{URIPATH:url}

[Shaju (Devassy)]

sorry for my ignorance. Trying now with the understated and simulator.

rule “CiscoFirepower”
when
has_field(“message”)
then
let message = to_string($message.message);
let splittraf = split(",", message);
set_field(“EventPriority”, splittraf[0]);
set_field(“ApplicationProtocol”, splittraf[1]);
set_field(“AccessControlRuleAction”, splittraf[2]);
set_field(“SrcIP”, splittraf[3]);
set_field(“DstIP”, splittraf[4]);
set_field(“IngressInterface”, splittraf[5]);
end

please help me to use
value split => “:” and field_split => “,” as now i am not getting the right values for fields

[Shaju (Devassy)]

sems the key_value is doing nothing, trying with simulator. seems like it’s firing but not parsing.

rule “key_value_parser”
when
has_field(“message”)
then
set_fields(
fields:
key_value(
value: to_string($message.message),
trim_value_chars: “”,
trim_key_chars:"",
delimiters:",",
ignore_empty_values: true,
kv_delimiters:":"
)
);
end

[gsmith]

Hello,

I might be able to help.
I’m not that good at GROK patterns with pipelines but maybe something like this?

rule “Batman ”
   when  true
then
   let mess = to_string($message.message);
   let parsed = grok(pattern: “%{FIRST_PATTERN}”,value: mess,only_named_captures: true);
   let parsed = grok(pattern: “%{SECOND_PATTERN}”,value: mess,only_named_captures: true);
   let parsed = grok(pattern: “%{THIRD_PATTERN}”,value: mess,only_named_captures: true);
   set_fields(parsed);
end

There are many examples in the forum to try out. The search box and tags will be very helpful.

Hope that helps

[gsmith]

I just noticed some of those fields you want look like mine.
This environment mainly uses REGX and some GROK.

Here is examples of some of mine. I use this for Cisco and Fortinet.

Example of DstIP

Results

[Shaju (Devassy)]

Thanks in loads gsmith for you reply and tips and sorry for my late response.
Had googled further looking out for answers why my pipeling rule “key_value_parser” ( last pasted ) didn’t work.

I found that i need to change the order under “System-Configuration-Manage Processors Configuration”. After changing it didn’t reflect immediately, somehow after an hour of two i saw it was parsing fine. Apprecitate you time taken to assist me. Sure your code will be of help in some way or other as i am still exploring and filling up graylog