SIEM Modernization and Optimization: Step 4 - Measure and Optimize

For anyone with a “set it and forget it” mindset to AI-powered SIEMs, keep reading. While SIEM optimization and ultra-modern SIEMs create huge gains in efficiency — without constant AI prompting — tracking and oversight becomes more important.  

CISOs and SOC leaders in charge of SIEM modernization need to incorporate stringent measurement and optimization processes to ensure risk levels are acceptable and continuously improving.

In our fourth and final post on modernizing SIEM for the AI era, we’ll dig into key performance indicators (KPIs) to focus on and drive measurement and optimization. But if you’re just tuning in, here’s what we’ve covered so far:

1. Assess the Data: Objectively evaluate your current data sources, tools, and processes to identify critical gaps and pain points.  

2. Define Goals: Set clear goals for your target architecture and align them with your organization's risk tolerance.

3. Implement Strategically: Phase rollout, beginning with a high-value, high-impact use case to secure buy-in across the organization.

Success Means More Than Increased Detections

If you’ve followed the previous steps in our SIEM modernization and optimization blueprint, that means you’ve established a unified data architecture (i.e., an observable data lake of all cyber-related information). This is what the AI model consumes to detect suspected events, including sophisticated attacks across multiple systems.  

Presumably, detection counts will go up. That’s good, as you’ll be more aware of potential network intrusions. However, focusing only on detection counts misses the goal of an ultra-modern SIEM: proactively contain attacks to systematically reduce risk.

Tracking reduction in critical incidents is a better indicator of success than detection counts. This KPI will prove that the new SIEM is effectively predicting and preventing attacks before they can be successful.

If critical incidents are going up, this may be an indication that:

1. Your organization is the target of yet more threats

2. Your AI models need tweaking  

Both can be true. But SOCs never want an “own goal.” Take the data point for what it is and hunt for where improvements can be made, from the data that feeds the AI, to the analysis and response workflows.

Know Your Mitigation Time vs. Average Breakout Time

Mitigation time is the time from detection to when an attack is contained. Breakout time is the time it takes a threat actor to succeed in a compromise attempt and establish an initial state of persistent access in your environment.  

Breakout time continuously shrinks, and threat actors’ use of AI has accelerated the decline in average breakout time. Recent estimates put average breakout time at less than an hour. Keeping up with this average is important, so focus delivering an average mitigation time less than this figure.

If mitigation time exceeds breakout time, this may be indication that:

1. Threats are continuing to increase the speed of successful attacks

2. Your AI model needs tweaking

Again, both can be true. Interrogate AI models to learn where to make the biggest gains in time-savings.  

Continue the Success of SIEM Modernization and Optimization

Tracking these metrics will help you to continuously refine AI models to be more efficient, the alerts more confident, and your SOC team more successful. Through optimization and maturity processes, you can ensure your ultra-modern SIEM is significantly lowering your risk exposure.

See the full blueprint for SIEM modernization in 4 Steps to Modernize Your SIEM for the AI Era.

‍