I’m trying to enrich my logs with tags such as directionality, or activities etc. It’s easy in the collectors, but I just want a dynamic field that I can add “authentication” and “sysmon” and “internal_to_external” or whatever I feel like addding.
Again, it’s easy in sidecar or the collector, but how do I do it in the pipeline? I’m trying to build a one-stop SIEM rule with windows events and I wanted to start adding my enrichment to it. I’ve found a few references to teh capability, but unless I borked it,
add_tag = “authentication”
You’re probably looking for set_field() / set_fields(). It’s also possible you’re looking for streams. Streams are effectively meta-data tags on messages, in which case it’s route_to_stream().
I had never thought of using Streams in the same sense as Tags. I was thinking more like a universal boolean search, so I could add additional details to each message:
Event 1: RDP Interactive Login success for user BlueTeamNinja (Add tags: RDP, Login Success)
Event 2: SSH Login success for user BlueTeamNinja (Add tags: SSH, Login Success)
So a chart of Successes by type, or Types by Success are as simply as searching for > SSH
I suppose Streams are the same, but that seems like more overhead to me.
A further thought - don’t streams store the messages separately?
A boolean tag of SSH, login_success, alert_data,suspicious_activity etc is a lot less disk than duplicating the same message in 5 streams of varying value.
You can use set_field() in the pipeline to add a “tags” field and whatever tag data you want. Below would create a new field called “TAG” and contain [“NEW-USER”, <target username from message>]
rule "AP2-WinSec-NewUser"
when
to_string($message.winlogbeat_event_id) == "4720"
then
let stufftotag= concat("NEW-USER", to_string($message.winlogbeat_event_data_TargetUserName));
set_field("TAG", stufftotag);
route_to_stream("P2-Alert");
end
on a side note I am writing up a framework for watching windows events. It’ not quite done/documented yet. I tried posting it up to the Graylog Marketplace but thats is broken Feel free to use what you want from the below link - if you have ideas and time to share them I am always looking for improvement.
From what the documentation says, streams are literally just tags on the messages, not separate storage. That’s why messages can be in multiple streams.
From what the documentation says, streams are literally just tags on the messages, not separate storage. That’s why messages can be in multiple streams.
but streams are the seperators for different indices - what makes the messages in multiple stream save multiple times if the streams are bound to different indices.
I think I’m back to my original stance and possibly a feature request. Both Index sets and Streams, their separators require planning and forethought overall. Whereas a simple tag, most universally seen as a set of Booleans with names (as the Beats collection
already uses) can be more or less ad hoc without affecting the overall architecture or strategy.
They also differ from String concats as they are seen as individual Booleans, whereas a string search would require left-wildcards, or overly complicated queries if they happen to be in a different order.
E.g.
Field: Extra info
Data: A B C D
To find ones containing (D or B) is harder than it has to be. With tags, it would be
As others say you can add a tag field with a pipeline (set_fields) or you can also use an extractor (replace with regular expression).
Personally I tag my events (atuhentication, network…) thanks to extractors.
However Graylog does not support multivalues fields so it’s a bit tricky if you need to add multiple tags to one event.