So I am trying to integrate Threat intel lookups into my Graylog using a Pipeline rule. Right now the setup is all firewalls are forwarding into Graylog, Grok pattern extraction (these are Cisco Firepower Firewalls). The builtin for AbuseCH, Spamhaus and TOR work perfect, no issues. When I set up a stage for OTX per information here I can watch my Process Buffer fill and then the journal starts to fill up with no end in sight to it. I remove the OTX lookups and everything goes back to normal. HEre is what I have as a rule:
rule "Threat Intelligence Lookups: OTX src" when has_field("src_addr") then let intel = otx_lookup_ip(to_string($message.src_addr)); // let intel = otx_lookup_domain(to_string($message.dns_question)) set_field("src_OTX_threat_indicated", intel.otx_threat_indicated); set_field("src_threat_ids", intel.otx_threat_ids); set_field("src_threat_names", intel.otx_threat_names); end
This is run against the All Messages Stream, but the firewalls are the only thing going into Graylog at this point. Any help would be appreciated.