What is the most effective way to parse logs with lots of optionnal fields?

Good morning, I am completely new to Graylog (or any SIEM really) and I was tasked to parse our firewall’s logs.

There are around 40 total fields with only 10 being there all the time so the 30+ others are optionnal.

I have been using one pipeline and GROK to parse the logs so I have a lot of ? because of all the optionnal fields.

Is this the way to do it or is there a more efficient way?

Thank you,

Hello && Welcome @FrenchToast

Easiest way I now of is using a different INPUT Raw/plaintext UDP/TCP this should limit you number of fields and then create the fields needed either with pipeline or extractor.

Here is a note on how I handled firewall traffic coming from PaloAlto firewalls:

1 Like

I just read that post :eyes:, Thankfully we don’t use PaloAlto :laughing: . @tmacgbay someone on discord was asking about dashboard with PA.

hmmm … maybe post mine for the contest…

1 Like