Workaround for CEF - Integer/Double

Graylog Central (peer support)
1

Hi!

I am sending CEF-logs from Fortigate-firewalls to graylog.
IN/OUT-bytes are sometimes bigger than the integer limits. This is already a known issue:

But: Do you have any idea on how to do a workaround to this? Do you have any idea for an alternative handling of the logs?

Currently, I am using a pipeline-rule to extract CEF:
let result = regex(("(CEF:.*)$"),to_string($message.message));

But there is:
WARN [MappedMessage] Could not transform CEF field [out] according to standard. Skipping.
java.lang.NumberFormatException: For input string: “2273997656”

Thank you for your thoughts
KPS

2

Hi!

I just found a “nasty” workaround:
My rule-order is: Pipeline->Extractor

Pipeline:

rule "Fortigate Process CEF for traffic"
	when
		contains(to_string($message.message), "CEF:0", true)
	then
        let result = regex(("(CEF:.*)$"),to_string($message.message));
        let pure_cef = to_string(result["0"]);
        // Remove in and out for later extraction from message field by EXTRACTOR
        let pure_cef = regex_replace("(.*)( out=[0-9]{1,10} )(.*)", pure_cef, "$1 $3");
        let pure_cef = regex_replace("(.*)( in=[0-9]{1,10} )(.*)", pure_cef, "$1 $3");
        set_fields(parse_cef(to_string(pure_cef), false));
        set_field("message", to_string(result["0"]));
end

And extractors from message-field later.

KPS

3

Hi!

I had to change the rules and go to “pipeline-only” to be able to convert to “double”:

rule "Fortigate Process CEF for traffic"
	when
		contains(to_string($message.message), "CEF:0", true)
	then
        let result = regex(("(CEF:.*)$"),to_string($message.message));
        let pure_cef = to_string(result["0"]);
        // Remove in and out for later extraction from message field by EXTRACTOR
        let pure_cef = regex_replace("(.*)( out=[0-9]{1,20} )(.*)", pure_cef, "$1 $3");
        let pure_cef = regex_replace("(.*)( in=[0-9]{1,20} )(.*)", pure_cef, "$1 $3");
        set_fields(parse_cef(to_string(pure_cef), false));
        set_field("message", to_string(result["0"]));
        // set_field("pure_cef", to_string(pure_cef));
end

rule "Fortigate Process Traffic Counters Full Sessions"
	when
		contains(to_string($message.message), " in=", true)
		AND contains(to_string($message.message), " out=", true)
		AND NOT contains(to_string($message.message), " FTNTFGTlogid=0000000020 ", true)
	then
	    let intotal = to_double(regex_replace("(.* in=)([0-9]{1,20})( .*)",to_string($message.message),"$2"));
	    let outtotal = to_double(regex_replace("(.* out=)([0-9]{1,20})( .*)",to_string($message.message),"$2"));
	    set_field("in", to_double(intotal));
	    set_field("out", to_double(outtotal));
end

rule "Fortigate Process Traffic Counters Partial Sessions"
	when
		contains(to_string($message.message), " in=", true)
		AND contains(to_string($message.message), " out=", true)
		AND contains(to_string($message.message), " FTNTFGTlogid=0000000020 ", true)
	then
	    let intotal = to_double(regex_replace("(.* in=)([0-9]{1,20})( .*)",to_string($message.message),"$2"));
	    let outtotal = to_double(regex_replace("(.* out=)([0-9]{1,20})( .*)",to_string($message.message),"$2"));
	    set_field("intotal", to_double(intotal));
	    set_field("outtotal", to_double(intotal));
	    let inbyte = to_double(regex_replace("(.* FTNTFGTrcvddelta=)([0-9]{1,20})(.*)",to_string($message.message),"$2"));
	    let outbyte = to_double(regex_replace("(.* FTNTFGTsentdelta=)([0-9]{1,20})(.*)",to_string($message.message),"$2"));
	    set_field("in", to_double(inbyte));
	    set_field("out", to_double(outbyte));
end
4

Hello @KPS

Nice I was wondering if you have seen the category in the forum below. It would be nice if you could write up something under pipelines there for future community members. This would give a better search later on. Also thank you for posting your resolution to the issue :slight_smile:

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.