Threatintel WHOIS - 50% of country code's and organizations are "N/A"

Graylog Add-ons
1

I’ve been using the threatintel plugin, to build out a SIEM product, for the consideration of our business.
Everything has been working fine, with the exception of WHOIS lookups.

At first, I was getting “N/A” for every country code and organization; this was simply due to TCP-43 not being open on our firewalls.

Now the rules are in place, what I’m seeing is partial-success, where around 50% of the lookups are being completed, but the other 50% are still showing as “N/A” (this is ignoring RFC 1918 addresses - these are ALL external addresses which I can manually look-up successfully).

I can’t see anything in the logs (prior to having port 43 open, I was seeing a lookup error, now there’s nothing).

Has this behavior been noted by anyone else? Is there anywhere specifically I should look? Or is this never likely to be viable. Since my OTX lookups are ALL working, it seems odd that there’s a timeout issue. My cluster load is also below 5% and this is a large datacenter, with a 400Mbps internet-out.

Real-world example;

Thanks,
Tom

2

Hej @tomjcollins85

my personal recommendation is - for country lookup - do not use the thread-intel plugin.

Why? Every lookup will create a WHOIS call and you might get rate limited (what might happen right now) just because of the amount of lookups you do.

If you want to have the country, let first the geoip-lookup plugin do the work. That is an offline call to the local database. You might only want to update the maxmind database on tuesday to have the latest version running.

Only if that lookup did not deliver a match, do the whois lookup. That will give you some additional performance.

/jd

3

Thank you, Jan.
That sounds good, and I think real-time WHOIS is something we can live without.

I’m already using the geolocation plugin - is there any way of generating a new ‘country’ field, based on the _geolocation field? Or do we need to stick to the visual map? It would be great to be able to quantify countries for display/reporting etc.

Many Thanks,
Tom

4

hej @tomjcollins85

if you just follow the steps in the documentation you will get that field without any additional work.

/jd

5

Is that possible on 2.1.3, at all? (noticed the line ‘Since version 2.2.0’, and my version only seems to create the _geolocation field)

Thanks,
Tom

6

Is that possible on 2.1.3, at all?

currently not possible. As you did not wrote your Version I had assumed you are on the latest.

Just to have it written - we have a thread-intel plugin for version 2.2.x … if that was the reason for not updating.

7

Thank you for all your help.

Upgrading to the latest version was attempted, but I just couldn’t get the web front-end to start up.
I will set up a test system and see if I can get any further.

Thanks again,
Tom