Doing Threat Intel the Hard Way - Part 1: Manual IOC Management

Part #1: Introduction to Manual IOC Management for Threat Intelligence

This is the first post of a series on manual management of IOCs for threat intelligence.

Threat Intelligence is a popular topic in security circles these days. Many organizations are now utilizing a threat feed that comes bundled with some other security product, such as McAfee’s GTI or IBM’s X-Force feeds. These feeds deliver indicators of compromise (IOCs) to their subscribers. Lots of products, notably SIEMs, have added support for some sort of integration with specific Threat Intelligence feeds or more generic imports via STIX/TAXII. Many shops are now hoping to take advantage of the large number of open source and free intelligence feeds available. Some are even investing in commercial intelligence feeds.

However, as many organizations quickly discover, without effective management of the Threat Intelligence Indicator Lifecycle, making effective use of this valuable information is nearly impossible. Today, an organization has two choices for managing IOC’s for Threat Intelligence, Threat Intelligence Management platforms such as Anomali, or a manual in-house management program.

Although I spend most of my time explaining and demonstrating The Anomali ThreatStream Threat Intelligence Management Platform to prospects, in this blog series, I am going to describe the steps required to set up a manual threat intelligence IOC lifecycle program for those who prefer the in-house approach.

Effective threat intelligence IOC management consists of six main functions or processes,

• Threat Intelligence Source selection
• Threat Intelligence Capture
• Threat Intelligence Processing
• Operationalizing Threat Intelligence
• Threat Intelligence Analysis
• Threat Intelligence Maintenance

Each of these processes requires consideration of multiple challenges and requires particular skillsets be present or contracted. We will explore each of them in detail over the course of a series of blog posts.

Source Selection

Source Selection is actually not the first step in setting up a manual threat intelligence program. Before any threat intelligence can be made useful, you must first have something against which to compare it. This will usually be some kind of log management system or SIEM technology collecting logs or other key information from security devices in your environment. Without this critical foundation, there is no way to correlate what is happening in your environment against the intelligence you are collecting and therefore no way to know when you are communicating with any of the malicious indicators you have identified. Choose carefully, as the limitations of the chosen solution may reduce your options when it comes time to integrate your threat intelligence.

Assuming you have an adequate solution in place, you are ready to select the intelligence sources from which you wish to collect. You can choose from free/open source feeds or you may purchase a feed from one of the several dozen vendors in the market today.

There are well over a hundred free or open source intelligence feeds available. Many of these feeds get their indicators from the same sources and report on the same indicators, creating large areas of overlap and duplication of indicator data. This is an important consideration, as too much overlap can negatively impact the later stages of the threat intelligence management process.

There are dozens of paid feeds available as well. Each has their own areas of focus, and costs vary widely. Although the quality of paid feeds is high, the cost of subscribing to multiple feeds can add up quickly. Careful attention should be paid to contract negotiations with feed vendors so that you are absolutely clear about which of their feeds you will have access to and which you will not. Another important consideration should be the methods supported for ingesting those feeds. A flexible API (Application Program Interface) would be an advantage in this instance, since you will be integrating each of these sources in-house.

Up next in the series: Capturing Threat Intelligence