CISCO ASA Severity Level

[markinhuszn]

Hello,
I have set up an input for my CISCO ASA logs and i am wondering if there is a way to change the severity level of the logs that get filed into the database? simply because of disk space consumption (i want to decrease it).
It is of my understanding that I need a “regular expression” in order to do this? if so can someone point me in the right direction?
I want to discard any syslog entries lower than “alert” and “emergency”.

[derPhlipsi (Philipp Ruland)]

Hey @markinhuszn,

@markinhuszn

It is of my understanding that I need a “regular expression” in order to do this?

Yes, this is one posibility. But there is a better one.

Have a look at Pipelines.
You could write a rule that checks if $message.level is higher than 1 and then uses drop_message(message: Message) to remove the current message from further processing.
Example:

rule "dropUnwantedLevels"
when
    $message.level > 1
then
    drop_message();
end

Greetings,
Philipp

[markinhuszn]

Thanks for your help
I have tried to implement your solution but I get prompted an error on the third line stating:

Any help would be appreciated. Thanks

[derPhlipsi (Philipp Ruland)]

Sorry, I missed that you’ll have to convert $message.level to an int first:

rule "dropUnwantedLevels"
when
    to_long($message.level) > 1
then
    drop_message();
end

[markinhuszn]

Thanks that went through
I have added the code, now im stuck with other configuration issues Im guessing.
I have All Messages going through the pipeline connection. I have selected the pipeline configuration you gave me but for some reason the messages are not going through that configuration. Any clues?

[derPhlipsi (Philipp Ruland)]

Change the rule to this:

rule "dropUnwantedLevels"
when
    true
then
    set_field(field: "PIPELINE_DEBUG", value: to_long($message.level))
end

And have a look at what the to_long() writes. I did not test the rule myself, so I only guessed that this should return an integer that you can check.

Greetings,
Philipp

[markinhuszn]

rule “dropUnwantedLevels” when true then set_field(field: “PIPELINE_DEBUG”, value: to_long($message.level)) end

Thanks for your response,
I have ran that code and it got logs. It created the field: PIPELINE_DEBUG with a value of 0.

Also: the logs I want to log are the ones from CISCO ASA:

So perhaps the reason it hasn’t logged anything with the previous pipeline is because it contains the message field whereas it should’ve been severity level?
just a guess

[jan (Jan Doberstein)]

what is your processing order? ( System > Configuration )

It should be

1. message filter chain
2. processing pipeline
3. geo-ip

the others are only needed if you use the inputs.

[markinhuszn]

Hey Jan,
Thanks for the response and help.
I fixed the message processors configuration to this:

still, it doesn’t seem to be working - I still get CISCO ASA logs with severity > 1

[markinhuszn]

wouldnt it just be easy to insert a regular expression on the Input that I actually want to filter instead of running all messages through a pipeline?

[derPhlipsi (Philipp Ruland)]

The regular expressions extracts information. It cannot drop messages.

You want to drop any message above level 1, correct?
That you’ll have to do with a pipeline function, since that is the only way to stop a message from being processed and indexed.

Can you show us what value is inside the level field atm? Is it numeric or is it a text like “Info”, “Warning”, “Critical”, etc.?

Greetings,
Philipp

[markinhuszn]

Its numeric
I’m getting a lot of 6 and 4’s

[derPhlipsi (Philipp Ruland)]

Well, that’s strange that the to_long() function turns a 4 to 0. This normally indicates a formatting exeption.

Doing a quick search I found this:

Graylog pipeline problem

Joechen, Sorry about that. If i change to_double to to_long, look like the above reply. The bandwidth_kb will be 0 for all case. when has_field(“spendtime”) then let time = to_double($message.spendtime); let size = to_double($message.bytes); let bandwidth = size / time ; let bandwidth_kb = to_long(bandwidth / 1024.0); set_field(“speed_bytes_sec”, bandwidth); set_field(“speed_kb_sec”, bandwidth_kb); end speed_bytes_sec 67548.1865284974 speed_kb_sec 0

Which Graylog version are you running? to_long() returning zero is a bug prior to Graylog 2.4.0

Greetings,
Philipp

[markinhuszn]

Hey,
I am using the latest version of Graylog. It is indeed a weird thing.

[derPhlipsi (Philipp Ruland)]

Ok, then @jochen will probably have some interest into this topic

The to_long() seems to still have an issue parsing numbers…

For reference:
https://github.com/Graylog2/graylog-plugin-pipeline-processor/pull/219

Greetings,
Philipp

[markinhuszn]

rule “dropUnwantedLevels” when to_long($message.level) > 1 then drop_message(); end

Im thinking perhaps, the reason its not display is because its not catching the messages:
Here is my inputs:

I only want to extract the messages from CISCO ASA Input
Here is my configuration:

Here is a sample message for CISCO ASA logs:

Here are the extractors:

Here is the pipeline:

Here is the pipeline rule:

[markinhuszn]

Hello, Any help? really stuck with machine constantly running out of memory :{

[jan (Jan Doberstein)]

you look into the field $message.level but the name in the screenshot give severity - change the pipeline rule to match the field name and it should work.

[markinhuszn]

my exactly thoughts, but this is what I get:


If I remove it as the error says:

It still dont output anything.

[jan (Jan Doberstein)]

you need to use $message.severity that will give you the content of the field severity.