Performance is often one of the biggest gripes I hear from Splunk users. Even after spending time carefully architecting a distributed search environment, running it on top-of-the-range hardware, and carefully assigning user permissions, Splunk searches can still often run painfully slowly.
This scenario is particularly true of security use-cases where disparate data sources are compared to identify malicious patterns of activity. Whilst these searches are invaluable, many organisations have to heavily restrain the number of these searches their SOC or threat intel teams can run.
And that’s a big problem, not least because it’s very likely you’ll miss critical security incidents as a result.
Here at Anomali we love Splunk. We not only develop a number of integrations for it, but also leverage it heavily to monitor our own applications and network. We too, however, face the same performance challenges as described above.
As part of our internal security posture, our security operations team analyse the large volume of data our network generates against the millions of indicators of compromise (IOC) curated in our ThreatStream product. It is a powerful way to identify threats immediately.
For a long time, we used our Commercial Splunk App to do this with great success. However, another key use-case for our sec-ops team involved identifying if threats had been seen historically. That is, if the team saw an IOC match our network data, they would then want to look historically to see if the same indicator had been seen before, perhaps days or months previously.
This was important as it would help build up a picture of the behaviour of any potential threat. In doing so, not only could a threat be stopped but it also allows for forensic analysis of what the threat could of potentially compromised.
With terabytes of data spanning years and millions of IOCs, this type forensic search was simply not possible in Splunk. A search like this would bring the best search node down very quickly.
Our pursuit of a workaround eventually led us to build Anomali Enterprise (AE), a product designed from the ground up to compare years of network data against millions of IOCs without breaking a sweat.
You can feed Anomali Enterprise from a wide variety of sources; from raw network events through to data stored in existing SIEM deployments, including Splunk. After AE has done the hard-work of identifying matches in real-time, you can either view the results in the native workflow or reingest the match data back into Splunk and your familiar workflows.
We’re here all week at Splunk .conf 2017 (stand G1) and will be ready to discuss how you can super-charge your Splunk security related use-cases using threat intelligence.