I spent days searching for a solution to the above. Graylog’s AWS plugin doesn’t work in this case unless you have your own bucket that FDR is dumping into, and Filebeat can’t read the input (likely because the data is stored in gz). So for those that want an actual solution that doesn’t involve “Just spend thousands per month on Splunk!”, here it is:
- Use Logstash with the s3 plugin. Example conf.d/fdr.conf:
input {
s3 {
access_key_id => "AKblahblahblahblah"
secret_access_key => "ThisIsNotTheSecretAccessKeyYouAreLookingFor"
bucket => "CrowdstrikeWillSellYouThis"
region => "us-some-region"
additional_settings => {
force_path_style => false
follow_redirects => false
}
}
}
output {
gelf {
host => "GraylogIPorHostname"
port => PortNumber
sender => "FDR"
}
}
No, gelf isn’t required. The challenge was never the output.
Also: Default FDR settings (no filters) will generate at least 5GB/day by itself, flooding Graylog with data every 5 minutes.
And finally, this data is in json format, so once the flood starts flooding, create an Extractor on the Message field, select JSON Extractor, and you should be good to go. You’ll likely have to create another extractor somewhere in order to get the Timestamp to work.