Example not working from Blog Post: Tapping Wires for Lean Security

[BlueTeamNinja (aBe)]

Hey All,

I am trying to use the example pipeline rule from the blogpost at Tapping Wires for Lean Security

This rule has a handful of errors on Graylog3 - starting from the “has” and the regex line at the bottom.
Any idea how I can update this for graylog3? Thanks.

rule "rewrite raw packetbeat DNS logs"

when
 
  has
  name);
 
  remove
  ip);
  remove
  set *field("dst* addr", $message.packetbeat <em>ip);</em>
 remove
  field("packetbeat_ip");
  set *field("dns* flags *authoritative", to* bool($message.packetbeat *dns* flags <em>authoritative));</em>
remove
  field("packetbeat *dns* flags_authoritative");
  set *field("dns* flags *recursion* allowed", to *bool($message.packetbeat* dns *flags* recursion <em>allowed));</em>
remove
  field("packetbeat *dns* flags *recursion* allowed");
  set *field("dns* flags *recursion* desired", to *bool($message.packetbeat* dns *flags* recursion <em>desired));</em>
remove
  field("packetbeat *dns* flags *recursion* desired");
  set *field("dns* flags *truncated* response", to *bool($message.packetbeat* dns *flags* truncated <em>response));</em>
remove
  field("packetbeat *dns* flags *truncated* response");
  code);
 remove
  class);
 remove
  type);
 remove
  code);
 remove
  port));
 remove
  set *field("src* port", to *long($message.packetbeat* client <em>port));</em>
remove
  field("packetbeat *client* port");

  // Remove fields we don't need or want.
 
  remove
  in");
  remove
  authorities");
  remove
  count");
  remove
  answers");
  remove
  direction");
  remove
  responsetime");
  remove
  error");
  remove
  transport");
 remove
  method");
 remove
  resource");
 remove
  status");
 remove
  type");
 remove
  query");
 remove
  disabled");
 remove
  additionals");
 remove_field("facility");

  // Remove trailing . if there is one
 let fix = regex("(.+?)\.?$", to
  question));
 set
  string($message.dns_question)));

end

[BlueTeamNinja (aBe)]

To answer my own question - I did a find/replace and a bit of massaging for my own needs on the example.

Screenshot - Denied Zone Transfer

Notes:

• The source is the requesting client (not the DNS server)
• I like upper-case letters
• I tried to remove all the extra packetbeat logs - I haven’t checked if wildcards work, I just copy/pasted and find/replaced
• The rest of the blog entry still works using type:dns in a search query.
• I’d like to highlight Issue: 5653 to greatly simplify reducing noise in pipelines!

I couldn’t get the formatting to work, so here it is on pastebin: Graylog-DNS-Packetbeat-Rule

[megan201296 (Megan)]

@BlueTeamNinja just some feedback on your Pastebin example, I’d recommend using the “rename_field” function as it’ll clean it up and reduce processing. For example, having

set_field("source", $message.packetbeat_client_ip);
remove_field("packetbeat_client_ip");

is the exact same as

rename_field("packetbeat_client_ip", "source");

Also would be careful about your usage of “source” as a field being set as source is a default field added in graylog specifying the source of the log. I use src_ip, but you could us source_ip, client, client_ip, etc.