Threat Intelligence Lookups Supported by Graylog

Hi Team,

We are planning to perform threat intel lookups at the graylog end . We have around 20 plus different graylog consoles for different customers.

In my understanding i just have to enable the Threat Intel Configuration in System/configurations part of graylog to enable the Threat Protection and then we can write pipeline rules to detect the Threats.

We are dealing with application logs so i will be creating separate fields for capturing the external Source and Url’s and then running threat intel on those fields .

I am aware of below threat Intel plugins existing in graylog please let me know if i am missing something :-
• Spamhaus DB
• Tor Exit Nodes
• Abuse.ch Ransomware

if there is any other smarter way or new plugins please let me know .

Any suggestions on Threat Intel Part would be appreciated .

Hi @GrAlog_learner!
if you have 20+ instances it will be a pain to keep all TI on the same level and consistent. My recommendation is this:

  1. collect all your TI-sources and grab IPs, Domains, Hashes, what ever type you want to search for. List those IoCs into a CSV with the corresponding source
  2. write a simple webserver which will search those CSVs for your keywords of interest. If there is a hit return the source of the TI. We used python. You might search for IOC lookup Graylog e.g.
  3. in each Graylog-Instance:
    3.1) create a data-adapter doing http-querries to your little server
    3.2) create a cache for the TI
    3.3) create a lookuptable for TI with the created dataadapter and cache
    3.4) Crate a rule similar to this.
rule "ioc_lookup"
when
  has_field("ioc_lookup_value")
then
  set_field("ioc_lookup_result", to_string(lookup_value(
    lookup_table:"ioc-lookup", 
    key:$message.ioc_lookup_value, 
    default:"false")
  ));
end

The name of the field “ioc_lookup_value” needs to be adjusted to your field(s) of interest.

  1. create another rule deleting the field ioc_lookup_result if it contains “false”
rule "ioc_lookup_cleanup"
when
has_field("ioc_lookup_result") &&
$message.ioc_lookup_result == to_string(false)
then
    remove_field("ioc_lookup_result");
end
  1. search for messages with “ioc_lookup_result”: _exists_:ioc_lookup_result and you will find all messages with IOCs labeled with the TI source.

With this approach you will have your TI separated from you Graylog. It will be easy to feed in new IOCs, since the little server can add them for all your 20+ instances.

1 Like

I’ll add to @ihe 's answer and share a similar (yet slightly simpler) approach :

  1. Locate an online IOC feed of interest, for instance : https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
  2. Create a bash script that pulls the IOC data (e.g. malicious IPs in the example above) from the feed and formats it to a clean CSV file (with no duplicates or empty lines)
  3. Create a cron job to automate the execution of the script, so that the list of IOCs gets refreshed on a regular basis
  4. Create a Graylog Lookup Table which is based on the IOC data in the CSV produced by the bash script.
  5. Use Graylog pipelines to compare the fields of interest that appear in your logs (e.g. IPs) to those in the IOC list via the lookup table created in step 4. And add an ioc_match field if there is a match.

I agree this is simpler. But it does not scale well with 20+ instances :wink:

Hi ,

We want to initiate the Threat Intelligence at graylog end .

We want to achieve end to end goal in at least one graylog first where are able to see the different fields that gets created in graylog from Threat intelligence point of view.

Could i get any help on achieving this.

Initially i would like to begin with otx lookup or ones which are available with graylog easily.

We are still not sure about the kinds of logs we will receive , mostly we want to rub external IP’s and Domains/url’s with the Threat Intel.

@ihe , @H2Cyber thanks for the suggestion. I will discuss with my team on both the suggestions but before that we want to achieve Threat Intel concept in one graylog starting from enabling of threat feeds to pipeline rules , Dashboards , Alerts , whatever possible with threat feed data.

Have a look at this awesome three-part blog posted by Recon Infosec which covers the use of Threat Intelligence with Graylog.

Hi @GrAlog_learner,

maybe including MISP as a common platform will help?
So you have the same IOC sources everywhere.