Security Operations Centers (SOCs) live on automation and threat intelligence. It’s common to lookup every login IP address to determine if it’s known malicious. And analysts want each indicator of compromise (IOC), like IPs, hostnames, URLs, and hash in an alert to be enriched. If the IOC is known bad, highlight it. Modern SIEMs, like Panther, make these tasks easy. But threat intel can be expensive and usually has API limits. Caching can reduce the pressure on API lookups. If an IOC was malicious 5 minutes ago, I don’t need to look it up again immediately. To make the most of your money you should cache your threat intel lookups!
In this post, I’ll walk you through using Panther’s DynamoDB Caching to reduce lookups to VirusTotal’s API.
Panther does the hard work of setting up and maintaining the caching server. Within your panther-analysis rule, you’re already authenticated and ready to go. All you have to do is call get_dictionary() and put_dictionary() as follows:
def get_vt_report(ioc: str, cache: bool = True) -> dict:
"""
Receives a IOC and returns true if VirusTotal reports
Args:
ioc: An IOC is a string that is an IP address, url, domain, or hash
cache: True if you want a cache
Returns:
Returns a VT report or an empty dictionary if no report was found.
"""
if cache:
cache_key = f"{CACHE_PREFIX}{ioc}"
cached_report = get_dictionary(cache_key)
if cached_report:
return cached_report
vt_report = _fetch_report(ioc)
vt_report = truncate_report(vt_report)
if cache:
put_dictionary(cache_key, vt_report)
set_key_expiration(cache_key, time.time() + ONE_DAY)
return vt_reportBefore this quick little optimization we readily exceeded our daily API limit. But caching reduced our utilization from around 10,000 queries/day to about 1000.
This strategy isn’t specific to Virustotal or Panther. If you automate correlation of login events to threat intelligence or enrich IOCs on alerts, you should cache. A similar strategy can be implemented on most RESTful API using a standalone Redis server or DynamoDB.
