Graylog Threat Intel Plugin Results - Stuck in Tail cave

We are running multiple graylog servers set-ups with a combinations of enterprise and community editions .
As part of the security initiatives , we have decided to spin up the Graylog Threat Intel Plugins by referring to the blog -

We have managed to see the malicious IP’s in the logs but the otx_lookup_ip is NOT flagging it as *threat_indicated:true * So , is there any OTX API Calls limitations in the community editions and what could be challenge when the same ip address when searched in the Open Threat Exchange (OTX) - IP Test lookup section on the graylog console shows the score with positive numbers ( non-zero) . For that matter , we would eager to know what fields from the lookup result would be cached for flagging it as threat_indicated as True ( “single_value”: 0,)

Steps to Reproduce the challenge :

  1. Create a grok pattern to find ONLY the public ip from the logs
  2. From the input , configure the extractor to match the grok pattern
  3. in the search section , you will seeing the new field getting populated as labeled in the step-2
  4. Follow & complete the steps mentioned in the link ( replace the src_addr / dst_addr fields in the rules with the names you have created in step-2 )
  5. In the search page , fields section - we will see the new additional fields cropping up once the pipeline rules matches from the logs . Fields like threat_indicated
  6. When we click on the threat_indicated values … it is all showing as Zero ( 0)
  7. When we handpick the known malicious ip’s from the same logs showing as Zero in the GUI and search using the test lookup option given the lookup_tables , it shows the value with Non Zero (6) which means it malicious to our understanding .

what are we missing ! GrayLog version is 3.0

Also , does the "Threat Intelligence Lookups:* rule take care of all these 3 feeds [ Ransomware IP] [Tor Exit Node List] [Spamhaus DROP] or should we write explicit pipeline rules .

Do comment if we are missing any additional pre-requisites here ( internet connectivity ports
java version , etc … )

when you run 3.0 - you should upgrade because of some bugs in that part of the product.

What would be preferred stable version you recommend Jan . Thanks for your response

he @Hari

always latest stable - what is currently 3.3

Thanks Jan , the configurations started working in 3.1 itself . we can close this thread

2020-06-15 07:09:06,813 WARN [OTXDataAdapter] - OTX IPv4 request for key <> failed: Response{protocol=http/1.1, code=400, message=Bad Request,
url=} - {}

graylog@graylog-graylog3-1:~$ java -version
openjdk version “1.8.0_232”
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

Internet connectivity : Curl to public URL’s are working

Configuratoin settings of OTX : /system/lookuptables/data_adapter/otx-api-ip/edit . We have generated a API key to see if the error messages gets fixed , but no luck . Even with the default settings we had this data adapter warnings . We have made this change based on the Jan’s recommendation from the older thread

So please help us with the break-fix . We can’t afford these warnings in the production environments

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.