Remove prefix in fieldnames

[admin86 (Eric)]

Hi guys,

I am currently started to working with CEF logs from FortiOS/Fortigate. Many of the fieldnames starting with FTNTFGT as prefix. I want to rename the fieldnames and delete the prefix.

example:
FTNTFGTdst_host → dst_host
FTNTFGTtunnelid → tunnelid

• Fortigate adding sometimes new fields, so it would be great if that’s “update-safe” maybe with regex?

I guess using the pipelines is the right way. Any suggestions for the right approach?

My old way is to not use CEF and use straight regex as extractors buts consuming a lot of cpu power :-/

Thanks in advance

[tmacgbay]

Pipelines are more efficient with regex plus you can narrow where the cleanup rule happens to only be messages that have FTNTFGT in them, then have a follow on stage work on breaking out fields. Best thought I can come up with.

[gsmith]

Hello && Welcome @admin86
I agree with @tmacgbay Pipeline /w regex is the way to go Im trying to clean up my extractors into one Pipeline.

We have a range of FortiGate firewalls (60, 80 100 200). What I have is REGEX extractors
Here are a few of them.

Example of regex Extractor.

EDIT:

@admin86

I want to rename the fieldnames and delete the prefix.

By default FortiGate sends Syslog’s, can I ask why you want to change them to CEF?

[gsmith]

I was labbing this out just with regex
FortiGate Logs both CEF and Syslog log types.

Following is an example log in CEF:

Feb 12 10:31:04 syslog-800c CEF:0|Fortinet|Fortigate|v5.6.0|37127|event:vpn negotiate success|3|FTNTFGTlogid=0101037127

Following is an example of my FG logs in Syslog :

<189>date=2022-06-27 time=18:26:52 devname=“gregs_pc” devid=“FGT60D8675309” logid=“0101037127” type=“traffic” subtype=“local”

As for pipeline you may need to specify each regular expression, also try looking into “lookup tables”