I am running Graylog 5.2.2 and i am using several lookup tables to VirusTotal api, to check source and destination ip’s for blacklisting.
It is possible to create a dashboard or similar and monitor the api calls to Virustotal for each data adaptor?
The reason i am asking this is that a connector is reaching the 500 calls limit in a few hours and makes no sense. I checked the config several times and i would like to monitor how many calls are the connectors doing.
I’ve had this same challenge myself. Graylog does keep a lot of metrics on all sorts of things. I recommend reading through the page here: Metrics
A metric that is useful for tracking when and how many times a given data adapter is used is gl_lookup_table_requests_count. Not that while the metric is titled “lookup_table_requests_count” the entry displayed is actually the data adapter ID (Object ID/OID). Here is an example of the metric data talked about above collected in prometheus and displayed in grafana:
Here is a sample grafana query for this: rate(gl_lookup_table_requests_count{}[$__rate_interval])
Unfortunately this won’t tell you what the api calls were or what log messages triggered them. One issue i ran into was that even though i had less than 500 unique items being queried to virus total, i was still reaching the limit. Caching should have prevented this but i was never able to troubleshoot it further.
One last thing you may have noticed, the metrics only show the oid (object id) of the data adatper and not the name. You’ll need to cross reference that oid in graylog, for example, the url: /system/lookuptables/data_adapter/6537f1d4c07c677b926aabaf will open graylog to the specific data adapter so you can see the name:
