Hi,
I am trying to truncate windows events to remove the explainatory text in the full_message field.
I am using nxlog to forward events to greaylog.
An example would be: 07/16/10 11:18:30 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4624
EventType=0
Type=Information
ComputerName=HIDDEN
TaskCategory=Logon
OpCode=Info
RecordNumber=6039
Keywords=Audit Success
Message=An account was successfully logged on.
…
This event is generated when a logon session is created. It is generated on the computer that was accessed.
The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).
The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.
The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
In this example I am trying to remove everything after "This event is generated "
I used an extractor to replace this text with a space but that created a new field with the other text duplicated.
My extractor regular expression was
(?:This event is generated)(.*$)
Now I am looking at pipelines. Something like this:
rule “full message truncate”
when
has_field(“full_message”)
then
let temp_field=regex("(?:This event is generated)(.*$)");
remove_from_stream(temp_field);
remove_field(temp_field);
end
This obviously does not work. (Expected type string for argument name but found RegexMatchResult in call to function remove_from_stream)
I am wondering am I going the right way to solve this problem and if so can anyone help?
I don’t work with regex often so it took me a bit to get it right. Here is what I came up with:
Add the following line to the INPUT section of your NXLog config:
Exec if $Message =~ /(?sU)(.+This event is generated)/ $Message = $1;
If you use Sidecar you can add it to the ‘Add verbatim configuration’ section of the INPUT config.
Basically this uses regex to match the entire message up to, and including, the text ‘This event is generated’. The regex match is stored in the $1 variable. The $Message is then replaced by the shortened message.
Thanks for your help but I could not get the nxlog configuration to work.
I did solve my issue though!
For anyone else facing this I created a regular expression expression extracter (System->Inputs->Manage Extractors) on the Input to cut the text and move it into a new field named redundant. Regular expression (?=This event is generated)(.*$).
Then in System -> Configurations I edited the “Message Processors Configuration” order to raise “Message Filter Chain” above “Pipeline Processor”.
I then created a pipeline to remove the new field created by the extractor .
rule “remove extracted_field”
when
has_field(“redundant”)
then
remove_field(“redundant”);
end
