Extract relevant messages based on a list of values

[Mat]

Hi !

I’ve been asked to create complex rules on the Graylog we already use, but I can’t find any starting point.

We use Opentext to store all our internal documentation. This Opentext is installed over IIS. We already collect IIS logs in Graylog and apply few basic rules.
Log entries in IIS specify the URL accessed, and, more importantly, the file identifiation number a user is accessing (fileID).

We would like to trigger an alert each time some of the most sensitive files are accessed. So we need to compare the log entry field “fileID” (I’m not sure this is the correct wording) against a long (100+) list of sensitive fileID.
We could write a long rule with 100+ “OR”, but it seems a painful way to go (and not very future-proof). I was wondering if we could compare the field against a file (or a set) containing all the sensitive fileID.

I’m an absolute newbie concerning graylog, and despite hours of searching (on this forum and on the web) the only clue I found is this topic : Rule matching against a long list of values? answer is quite short and outdated.

I’m sure a detailed anwser is waiting for me somewhere, but I don’t know the correct wording that could lead me to this answer. Better, if someone have an answer, or specific documentation, it will help me a lot

Thanks a lot

[derPhlipsi (Philipp Ruland)]

Hey @Mat,

have a look at http://docs.graylog.org/en/2.4/pages/lookuptables.html

This is exactly what you need.

Little hint list:

1. Create a lookup table with a CSV file (or even better a HTTP endpoint when you have one you could query to get the most up-to-date data) that has all your sensitive fileIDs.
2. Create a pipeline with a rule that sets a field like isSensitiveFile = true of a fileID matches one listed in your lookup table. Example:

rule "isSensitiveFile"
when
    lookup_value("sensitiveFileIDs", $message.fileID) != null //Or select a field that you can check here as single value return in the lookup table for more granular matching
then
    set_field(field: "isSensitiveFile", value: true);
end

3. Create an alert that looks for this isSensitiveFile field and use the backlog to put more specific information in the notification text.

Greetings,
Philipp

[Mat]

Thank you so much Philipp. It seems it is exactly what I was looking for !
I’ll try to implement it right away