Math between fields

Graylog Central (peer support)
,
1

I extracted 2x fields - size and time. Now I need to divide one on another and alert the result if threshold is reached. Could you please advice how to do math between fields?

2

Graylog currently doesn’t support running ad-hoc mutations of indexed data, but you can use processing pipeline rules to run calculations on messages and store the result into new fields, see http://docs.graylog.org/en/2.2/pages/pipelines.html for details.

3

Thank you for your advice, jochen.
How can I create a rule to divide one field on another. I didn’t find that in docs. Described functions don’t do that.

Thank you !

4

You can use basic arithmetic operations as demonstrated in this test case:

Make sure to cast the fields to the proper type (e. g. with to_double()) before doing any arithmetics.

5

So I created the pipeline rule:

rule "WRITE bandwidth"
when
  has_field("totaltime") && has_field("sentbytes")
then
    let time = to_double($message.totaltime);
    let size = to_double($message.sentbytes);
    let bandwidth = size / time;
    set_field("wrbandwidth", bandwidth);
    route_to_stream(name: "All WRITE stream");
end

wrbandwidth field has be be created now. Did I catch you explanation correctly?

How to use new field to create a graph in a dashboard?

Sorry for the newbie questions. I’m using graylog for few hours but need to do the tasks ASAP.
Thanks for your support !

6

Yes.

Just like any other numeric field: http://docs.graylog.org/en/2.2/pages/dashboards.html#examples

7

I created the rule described above but the new field wrbandwidth appeared only after a while.
Next I created couple of more pipelines with new fields calculated by rules (read_bandwidth and writhe_bandwidth). After half an hour new fields are still not visible.

How long it takes to get new field visible in a search?
Can we speed up that?

8

Finally both new fields appeared. But the questions are still the same.

How long it takes to get new field visible in a search?
Can we speed up that?

9

This depends on your setup and a lot of parameters, but usually it’s a few seconds.

10

Only new messages, which run through the processing pipeline, will have the new fields.

11

I got the idea, thanks.

Everything works just fine now.
I really love Graylog !

Jochen, thank you for your support !!!

12

Hello all,

I’m replying to this old post as it seams doesn’t work for me. I created a pipeline rule to calculate the upload speed based on extracted fields from IIS logs (cs_bytes & response_time_msec).

rule "WRITE UploadSpeed"
when
  has_field("cs_bytes") && has_field("response_time_msec")
then
    let time = to_double($message.response_time_msec)/1000.00;
    let size = to_double($message.cs_bytes);
    let bandwidth = size / time;
    set_field("UploadSpeed", bandwidth);
    route_to_stream(name: "Web logs");
end

My problem is that the the criteria never come back as true. I used the “Web Logs” stream as the Pipeline connection to be able to use the extracted fields, but this didn’t help.

I wonder is this is something was suitable for older Graylog version (I’m using Graylog v2.4) or it is something I’m doing wrong?

Thanks in advance…
Mohamed

13

Do these fields exist when the pipeline rules are running? Are they created by some extractors?

Check the order of message processors in your Graylog cluster on the System/Configurations page.

14

Hi @jochen

My Message Processors Configuration was in the following order.

# Processor Status
1 AWS Instance Name Lookup active
2 GeoIP Resolver active
3 Pipeline Processor active
4 Message Filter Chain active

I pushed the Pipeline processor to be #4, and will try again

Thanks you.
Mohamed

15

Hi @jochen

It works :slight_smile: Thank you for the tip.

Thank you
Mohamed