Functions not working

Graylog Central (peer support)
,
1

Dear All,

Can someone explain to me why the following does not work

I have a function in a pipeline at stage 0 with two other functions with different matching criteria

Here is the function

rule "detect powershell-excel"
when
  // detects powershell executing from Excel
  has_field("sysmon_event_id") AND
  has_field("sysmon_data_process") AND
  has_field("sysmon_cmd_parent_file") AND
  contains("1",to_string($message.sysmon_event_id)) AND
  contains("C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe", to_string($message.sysmon_data_process)) AND
  contains("EXCEL.EXE", to_string($message.sysmon_cmd_parent_file))
then
  set_field("MitreRef", "T1086");
end

In the logs I have the following fields

image
image.png832×733 30.6 KB

I have also tried rotating the indexes but to no avail.

Why does it not set the MitreRef field?

I have other rules that set the same field for other conditions that work.

Any help appreciated.

Jake

2

I have also tried it with the path escaped

image
image.png884×194 9.26 KB

3

Do the fields (“sysmon_event_id”, “sysmon_data_process”, “sysmon_cmd_parent_file”) exist when the pipeline rule is being executed?

You could check this using the debug() function.

4

Hi Jochen,

I just want to make sure that I put debug in correctly as shown below

image
image.png890×199 9.46 KB

So if the fields exist in the log message, I should see a log entry in /var/log/graylog/server.log.

If I don’t see a message then the fields do not yet exist?

Am I correct?

Cheers

Jake

5

If you don’t see the debug messages in the logs of the Graylog node, then the condition in the when block wasn’t true. You’re checking more than just for the existence of these message fields in the when block of that rule.

6

Hi Jochen,

I changed my conditions to a simpler one and the pipeline works as I get debug messages in the log

image
image.png1160×130 5.18 KB

So it is my condition that is wrong!!

Cheers

Jake

7

Hi Jochen,

Within a function does $message refer to all fields in a message?

For example if I have a generated field such as sysmon_cmd_event, do I simply refer to it as $message.sysmon_cmd_event?

Cheers

Jake

8

$message is a reference to the currently processed message and you can access any message field via that reference:

$message.foobar // Message field "foobar"
$message.`foo-baz` // Message field "foo-baz"
9

Hi Jochen,

In my function i had this

when
// detects powershell executing from Excel
has_field(“sysmon_event_id”) AND
has_field(“sysmon_data_process”) AND
has_field(“sysmon_cmd_parent_file”) AND
contains(“1”,to_string($message.sysmon_event_id)) AND
contains(“C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe”, to_string($message.sysmon_data_process)) AND
contains(“EXCEL.EXE”, to_string($message.sysmon_cmd_parent_file))
then
set_field(“Ref”, “T1090”);

end

Now I know that the fields are present because if i do something like

when
has_field(“CustomerID”) AND contains(to_string($message.Channel), “Microsoft-Windows-Sysmon”)
then
let debug_message = concat("Field present ", to_string($message.sysmon_cmd_parent_file));
debug(debug_message);
let debug_message2 = concat("Field present ", to_string($message.sysmon_data_process));
debug(debug_message2);
let debug_message3 = concat("Field present ", to_string($message.sysmon_event_id));
debug(debug_message3);
end

I get the following

image
image.png1160×130 5.18 KB

So why doesn’t the above trigger? The pipeline stage is set to at least one rule must be matched and there are three rules in total. One of them is the very specific one that you help me create the other day.

Cheers

Jake

10

Because neither message fulfills the condition in the when clause of your rule.

11

Hi Jochen,

Forgive my lack of understanding. If I look at below, i think it should match

sysmon-1

I am assuming that EXCEL.exe would match sysmon_cmd_parent_file field unless contains is an explicit match.

Cheers

Jake

12

Hi Jochen,

I have confirmed that “contains” requires an explicit match. Is there a way to get it to match on part of a string?

Cheers

Jake

13

The contains() function does not require an exact match of the complete string.

contains(value: string, search: string, [ignore_case: boolean])

Checks if value contains search, optionally ignoring the case of the search pattern.

14

Hi Jochen,

It worked when I change it to the full path, but doesn’t work when i set to excel.exe or EXCEL.EXE.

I wonder if this is the reason why

contains(“C:\Program Files\Microsoft Office\root\Office16\EXCEL.EXE”, to_string($message.sysmon_cmd_parent_file))

Notice the space between Microsoft and Office could this cause the match to be terminated and so it never matches?

Cheers

Jake

15

The strings in the pipeline rule language are basically Java strings ().
This means that back slashes have to be escaped ( "\" → "\\").

For example, the following function call returns true:

contains("Foo Bar\\Baz", "Bar\\Baz");

EDIT: Maybe you should check the argument order for the contains() function in your pipeline rule. :wink:

No, the contains() function doesn’t handle whitespace characters any different than other characters.

16

Hi Jochen,

so why does it not match then?

If we have the string “C:\Program Files\Microsoft Office\root\Office16\EXCEL.EXE” it should match on either excel.exe or EXCEL.EXE?

How can the argument order be wrong if matches on the full path?

Unless, I am being very silly here :frowning:

17

Does it have to match the order of appearance in the log file?

19

Did you do this? :wink:

20

Just think about in which case the contains() function would return true, even if the argument order is wrong. :roll_eyes:

21

Hi Jochen,

How should the order be:

has_field(“field-x”) AND
has_field(“field-y”) AND
has_field(“field-z”) AND
contains(“match for x”,to_string($message.field-x)) AND
contains(“match for y”, to_string($message.field-y)) AND
contains(“match for z", to_string($message.field-z))

or is it meant to be

has_field(“field-x”) AND
has_field(“field-y”) AND
has_field(“field-z”) AND
contains(“match for x”,to_string($message.field-z)) AND
contains(“match for y”, to_string($message.field-y)) AND
contains(“match for z", to_string($message.field-x))

Cheers

Jake