Fields of the correlated events in the alert notification

[kaaroten]

Hi,
I’m following a private project and I’m quite stuck on one of the requirements:

We are receiving Windows NXlog from a management workstation and other Windows clients and we are trying to set a couple of alerts every time a login or a logout is performed through RDP

For every logout, 1 log is received from the source, but for every login, we receive 7 logs within 2/3 seconds:

• 4 x “Successful Login”
• 3 x “A user disconnected”

in no particular order (it changes every time) and I think it’s the authentication method’s fault, but I can’t be sure, since I haven’t permissions to log on appliances other than the graylog one.

The rules we wanted to enforce were:

• keep the first login in the chain and ignore the others
  which we solved by setting 1 backlog and 5 seconds of grace period in the notifications rules
• keep 1 logout, but only if there aren’t any logins within a 5 seconds period
  which would be easy enough per se, the correlation engine could work this out in no time, but the email should contain some of the fields of the message that triggered the rule.

Overview

Graylog 4.0.5+d95b909 (Debian 11.0.9.1 on Linux 4.19.0-14-amd64)

Event Definition (Windows - Account Logout (RDP))

• Event condition
  • Condition tipe: Filter & Aggregation
  • Filter
    • search query: “source:<name> AND EventID:4634 AND NOT TargetUserName:SYSTEM”
    • stream: “MGMT Workstation”
    • search in the last “10 seconds”
    • execute search every “10 seconds”
    • Create Events for Definition if… “Filter has results”
• No Notifications

Event Definition (Windows - Account Logout (correlation)

• Event condition
  • Condition tipe: Event Correlation
  • Event Correlation Rules
    • search in the last “10 seconds”
    • execute search every “10 seconds”
    • event #1
      • event from “Windows - Account Logout (RDP)”
      • should occur at least “1 time”
      • followed by:
    • event #2
      • event from “Windows - Account Login (RDP)”
      • should not occur in the next “5 seconds”
• Notification
  • [Email] Windows - logout event
  • message backlog (1)

Notification ([Email] Windows - logout event)

• Notification Type “Email Notification”
• Sender “graylog@server.local”
• Subject “Graylog event notification: ${event_definition_title}”
• Body Template:

<b>${event.message}</b>${if backlog}${foreach backlog message}
Message ID: https://graylog.server.local/messages/${message.index}/${message.id}
Timestamp: ${message.fields.DateTime}

Sistema: ${message.source}
Utente: ${message.fields.TargetUserName}


${end}
${else}<i>- no backlog -</i>
${end}

DateTime is a custom field created by an extractor.

Question

we need: Message ID ({message.index}/{message.id}), Timestamp ({message.fields.DateTime}**), `Sistema` (**{message.source}) and Utente (${message.fields.TargetUserName}) from messages that trigger the “Windows - Account Logout (RDP)” rule.

I didn’t quite understand the full potential of the “Custom Fileds” section and I found myself reading errors like:

2021-04-11T19:16:57.334+02:00 ERROR [TemplateFieldValueProvider] No value found for variable "message.source" in template "${message.source}"
2021-04-11T19:17:02.785+02:00 WARN  [CorrelationEventProcessor] Event <###> has no value for field <Sistema>

in the “server.log” without knowing where to watch to start troubleshooting.

Is it possible to use the “Custom Fields” section in the event definition to solve this problem? If yes, should I create custom fields on the base rule or in the correlation rule? Should I mark them as keys?
Is there any other way to retrieve the messages of the events triggering the correlation in the notification?

Thanks,

[shoothub]

Maybe problem is with correlation. Do you have same with `${message.source} problem without correlation?

FYI:
Not very known, but you can also also use same technique in message subject, for example I use this:
User ${foreach backlog message}${message.fields.username}${end} connected to switch ${foreach backlog message}${message.source}${end} from IP ${foreach backlog message}${message.fields.src_ip}${end}
It’s definitely better than only subject with alert name.

[gsmith]

@kaaroten

Hello,

What kind of INPUT are you using to ingest the logs for Windows?

I have found that using GELF was the right INPUT for us since were also getting alerts for user/s using RDP sessions. We created separate streams as shown below which allowed us to use special fields to cut out the white noise from Windows.

From there our Event Definitions/Notification have all the fields needed to do what you want to do without using Regular expression or GROK patterens.

Here is an example of our Notification configuration.

${if backlog}
--- [Backlog] ------------------------------------
Last messages accounting for this alert:
${foreach backlog message}
User:             ${message.fields.TargetUserName}
WorkStaion Name:  ${message.fields.WorkstationName}
Event Time:       ${message.fields.EventReceivedTime}
Source:           ${message.source}
Logon Type:       ${message.LogonType}
User SID:         ${message.SubjectUserSid}
Account Name:     ${message.AccountName}
Account Type:     ${message.AccountType}
--- [Backlog End] ------------------------------------

At times this does create a lot of fields.

Example of received message.

Hope this helps

[kaaroten]

Hi @gsmith
We are using GELF UDP for windows events

and use a single stream based on the source of the messages

---

Yesterday someone accessed the MGMT station and 7 events were generated

Events roughly translated from Italian

• [2021-04-17 14:08:59.000 +02:00] Account Access Succeeded
  EventID:4624
• [2021-04-17 14:08:59.000 +02:00] An Account Has Been Disconnected
  EventID:4634
• [2021-04-17 14:08:59.000 +02:00] Account Access Succeeded
  EventID:4624
• [2021-04-17 14:09:03.000 +02:00] An Account Has Been Disconnected
  EventID:4634
• [2021-04-17 14:09:03.000 +02:00] Account Access Succeeded
  EventID:4624
• [2021-04-17 14:09:03.000 +02:00] An Account Has Been Disconnected
  EventID:4634
• [2021-04-17 14:09:03.000 +02:00] Account Access Succeeded
  EventID:4624

All this from the action of loggin in through RDP.

I don’t want to receive an email telling me “Someone has disconnected” every time someone logs in.

Maybe it could help to know that the authentication process is made using NTLMv2

[gsmith]

@kaaroten

Thank you for the pictures, this helps a lot. We also had the same problem thats why you see in my post with different RDP streams. Have you tried to add another RULE for your stream.Maybe something like this, Make sure all rules must match.

That would filter out your disconnects.

The EventID 4624 is for An account was successfully logged on.
The EventID 22 is for “Remote Desktop Services: Session logon succeeded:”

Hope that helps

[kaaroten]

Hi @gsmith ,
thing is: I don’t need to separate the streams.
What is happening here is that, when someone logs in, among the User logged on messages there are User logged off messages, too.
Also, I recently managed to separate RDP from normal logins, but the problem remains:
when I login directly from the workstation. mixed logout and login events are sent to graylog (7 in total) and if I want to create separate alerts I need to sort out logouts generated during the login process.

[gsmith]

@kaaroten

Hello,

@kaaroten

thing is: I don’t need to separate the streams.

Yes you are correct.

@kaaroten

mixed logout and login events are sent to graylog

Maybe I have not explained myself very well.
On your stream, you would need to create another rule to filter out either logout and login.
For example:
Rule #1 use

EventID must match exactly 4625 ( i.e. Unknown user name or bad password.)

So any EventID with 4625 would be in that stream. If your getting other messages in that stream then what you need, add more rules. I gave an example above if that makes sense.
Here is an example of what I’m using. Remember Windows can use the same EventID for different events. Here is an example of mine using EventID 4624 ( User logged on).

Hope that helps.

[kaaroten]

Hi @gsmith ,
for starters I apologize for taking so long between answers: I don’t put my hands on the system very often.

Let me show you what I mean:

Here are Logins (upper) and Logouts (lower) side by side in the same timeline.

To the right you can see events that are generated during an actual login, to the right the actual logout.

When it comes to generate alerts for logouts, I receive an alert for the one event on the right (1100 hours circa in this example) but I also receive one for the 3 events on the left (0920 hours circa), which I don’t want because they are part of an “actual login” process (I am not aware of the reason for this behavior, my only guess is that the login process is complicated and requires more passages)

In the first post of this topic i explained what rule I tried to create to avoid unwanted alerts, but it didn’t work.
The rule was something like:
event from “Account Logout” should occur at least 1 time, followed by “Account Login” should NOT occur in the next 5 seconds

Strange behavior aside, can you help me solve this problem?

Thanks

[gsmith]

Hello,

Thank you for more details on the issue.
Adjusting stream rules to filter out only what you need would probably be ideal in this situation. Using two different streams. One for Logon and the other for disconnects would reduce the “white noise”. If you havent dont that already I would suggest trying out to see if it works for you. Then you can simplify you Event Definition, as shown below.
I have examples above that might help also.

Maybe try something like this, adjust this to your own needs, the following is just for failed logons. works for me without a problem.

Failed logon

Summary of my Event Definition:

Hope that helps.

[kaaroten]

Hi,
it helps, per se… so thank you very much.

Anyways, I’m trying to avoid a little thing with the output of the notifications, which I asked in another post in the forum.

I guess I’ll keep you posted