Truncate Windows Events


(brian) #1

Hi,
I am trying to truncate windows events to remove the explainatory text in the full_message field.
I am using nxlog to forward events to greaylog.
An example would be:
07/16/10 11:18:30 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4624
EventType=0
Type=Information
ComputerName=HIDDEN
TaskCategory=Logon
OpCode=Info
RecordNumber=6039
Keywords=Audit Success
Message=An account was successfully logged on.

This event is generated when a logon session is created. It is generated on the computer that was accessed.
The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).
The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.
The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

In this example I am trying to remove everything after "This event is generated "
I used an extractor to replace this text with a space but that created a new field with the other text duplicated.
My extractor regular expression was
(?:This event is generated)(.*$)
Now I am looking at pipelines. Something like this:

rule “full message truncate”
when
has_field(“full_message”)
then
let temp_field=regex("(?:This event is generated)(.*$)");
remove_from_stream(temp_field);
remove_field(temp_field);
end

This obviously does not work. (Expected type string for argument name but found RegexMatchResult in call to function remove_from_stream)
I am wondering am I going the right way to solve this problem and if so can anyone help?


(Jan Doberstein) #2

your rule does not work on any field.

  • you define regex as variable
  • this variable is then used to with two functions

you need to apply the regex pattern on a field ( http://docs.graylog.org/en/2.4/pages/pipelines/functions.html#regex ).


(brian) #3

Hi Jan,

Thank you for the reply.
I can see even if I apply the regex to a field, it only returns true or false?

Is it possible to use pipelines to remove unwanted text from a field?

(I was able to cut and extract the unwanted text to another field using extractors but was not able to delete the created field after!)


(Roger M) #4

I don’t work with regex often so it took me a bit to get it right. Here is what I came up with:

Add the following line to the INPUT section of your NXLog config:

Exec if $Message =~ /(?sU)(.+This event is generated)/ $Message = $1;

If you use Sidecar you can add it to the ‘Add verbatim configuration’ section of the INPUT config.

Basically this uses regex to match the entire message up to, and including, the text ‘This event is generated’. The regex match is stored in the $1 variable. The $Message is then replaced by the shortened message.

Hope this helps!


(brian) #5

Hi Roger,

Thanks for your help but I could not get the nxlog configuration to work.
I did solve my issue though!
For anyone else facing this I created a regular expression expression extracter (System->Inputs->Manage Extractors) on the Input to cut the text and move it into a new field named redundant. Regular expression (?=This event is generated)(.*$).
Then in System -> Configurations I edited the “Message Processors Configuration” order to raise “Message Filter Chain” above “Pipeline Processor”.
I then created a pipeline to remove the new field created by the extractor .

rule “remove extracted_field”
when
has_field(“redundant”)
then
remove_field(“redundant”);
end


(Roger M) #6

I’m sorry that didn’t work for you. For clarity I’m posting a copy of my entire input section.

  <Input 5ade305faa83a6186b3b7d63>
      	Module im_msvistalog
      	PollInterval 1
      	SavePos	True
      	ReadFromLast True
      	Exec $type = "Windows";
      	Exec if $EventID IN (4662) drop();
      	Exec if $EventID IN (30803) drop();
      	Exec if $SourceName in "iBossADPlugin" drop();
      	Exec if $Message =~ /(?sU)(.+This event is generated)/ $Message = $1;
     </Input>

Hope that helps.


(system) #7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.