Fortigate field mapping problem


I have syslog input in place to manage Fortigate firewall logs.
When log message has url with “=” all goes to sh…t. Graylog then parses using key=value and soon we have over 1000 fields…

example of url-field:
url: /CI0/01000170b4053790-f15ccb89-9c83-4f9d-a080-5fd73f0ace10-000000/sD7nfaNYjGkkaqRCxhU1wQmc9R1aHbs9XL2GFMpl_Lo=131

Above makes, IN ADDITION, field sD7nfaNYjGkkaqRCxhU1wQmc9R1aHbs9XL2GFMpl_Lo with value 131
url from the message-field:

I’ve tried using CEF-input, but it fails at the beginning to this:
java.lang.IllegalStateException: Could not parse timestamp. '<143>Mar 09 2020 15:52:38'

Suggestions for the best fix?
I was thinking of using pipeline, any thoughts? I would like to fix the message, not delete all messages with url…

Thank You in advance :slight_smile:

he @laakkus

I would use a syslog or raw input and do parsing like written in this issue:

Maybe you need to adopt that.

Tried with CEF UDP. need to modify somehow because graylog logs fill up with these now:

2020-03-11T11:45:00.576+02:00 WARN [MappedMessage] Could not transform CEF field [in] according to standard. Skipping. java.lang.NumberFormatException: For input string: "15787509737" at java.lang.NumberFormatException.forInputString( ~[?:1.8.0_242] at java.lang.Integer.parseInt( ~[?:1.8.0_242] at java.lang.Integer.parseInt( ~[?:1.8.0_242] at org.graylog.plugins.cef.parser.CEFMapping.convertInteger( ~[graylog.jar:?]

Does it try to parse it as INT, with mapping api I get type long…
"in" : { "type" : "long" }

TCP -formats don’t work from Fortigate. fails parsing.

Maybe I need to try rawUDP (like now) but with Fortigates sending “syslog”. Then modify the URLs in logged messages before parsing k=v pairs (these with pipeline)…
MAYBE then I will not have graylog server logs without filling with warnings AND the number of fields remains sane.

There seems to be a bug open for the “Warnings in CEF parser” -part of my post:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.