How Threat Hunting Can Evolve Your Detection Capabilities

Highly motivated threat actors continue to evolve their tactics, techniques and procedures (TTPs) to better evade traditional defenses and hide in the noise of your environment. Organizations must evolve their defenses and step past relying solely on traditional defenses and maximize their visibility to incorporate proactive detection methodologies — the prime example being threat hunting, which is the process of searching your environment for threat activity based on analysis of attacker tactics.

Growing businesses that neglect advanced detection methodologies such as threat hunting may more easily find themselves the victim of security breaches and the subject of headline news. While traditional defense tools are a necessity for an organization to prevent common malware infections, they often fall short in detecting and preventing more advanced and persistent adversaries — a risk that organizations of all sizes face. The addition of a threat hunting program creates an umbrella over the first-line defenses to both improve and supplement those capabilities, detecting otherwise unidentified adversaries.

In this post, we’ll explore the first steps in evolving your defenses and expanding proactive detection measures to better secure your environment and help mitigate the impact of potential security breaches. We break the post down into the following sections:

Threat Hunting 101 — the basics of proactive detection methodologies

Threat Hunting Framework — a framework for establishing a threat hunting program in your environment

Threat Hunting 101

The concept of threat hunting has a fairly recognized and agreed upon meaning of a proactive, repeatable and structured process of searching enterprises for threat activity that evades current detection capabilities. Some key tenets of this definition worth highlighting are:

Structured and repeatable: You can surely win a game of darts by closing your eyes and hitting a bullseye, but there are better ways to increase your chances of success. An efficient threat hunting methodology with the highest success potential is a well-planned and organized process in which you collect data, analyze that data and use lessons learned to improve the collection and analysis in the future. Threat hunting is a repeatable process that must evolve over time.

Threat focused: The end goal of threat hunting is the discovery and remediation of malicious threat activity in your environment, which helps to lower your enterprise risk. With this goal, every hunt should have an explicit purpose of identifying threat behavior. While network anomalies can be a symptom of threat activity, they are a small component of the larger objective that might be harder to diagnose than targeted analytics based on adversary techniques.

Enterprise-wide: You need to consider the entire ecosystem of the enterprise (endpoint, network, logs) when you base your hunt methodologies on attacker TTPs. Assume that there are legacy, unmanaged or unknown devices on the network that you do not know about and plan for those too. Consider network device compromises, IOT compromise and gaps in network security posture.

Beyond current capabilities: Threat hunting will result in the use of people, process or technology beyond the capacities of a traditional SOC, which is often reactive instead of proactive. Building a threat hunting program could require the addition of dedicated time, personnel or technologies to perform hunt-based activities.

Across organizations, the specific implementation of this concept can vary widely but still result in proactively detecting threats. It is important to recognize that not every hunt program will look or feel the same, nor will require the same level of effort. Start small and improve!

Threat Hunting Framework

While every threat hunting program will look different, there are frameworks you can leverage to begin exploring what works best for your organization. What follows is one such framework that you can leverage as a starting point for jumping into threat hunting.

1. Visibility

Robust visibility is the bedrock for a successful hunt. Limited visibility can skew your data and degrade confidence in your results. Simply put, when you increase your visibility into network, endpoint and log data, you simultaneously increase the effectiveness of your hunt.

Actions in this step might include deploying EDR agents, network visibility sensors and improving log aggregation. It’s also helpful to develop a data pipeline that will perform data enrichment. This could include adding IP address geolocation, operationalizing threat intelligence feeds and parsing data into standardized fields.

Additionally, after the result of the hunt engagement, the hunt team will work to gain additional visibility into areas that were lacking or newly identified network segments. This process should be always ongoing — making sure the entire security organization has the visibility it needs to properly secure and defend the organization.

2. Intelligence Analysis

During this phase, the hunt team will work to identify potential threats to the environment to gather reporting, insights and intelligence on those threats. Additionally, the team should study the network and work with key stakeholders to better understand the environment to prepare for execution of the hunt.

Gigamon recommends doing the following during this phase:

Use an attacker methodology such as MITRE’s ATT&CK framework to aid in adversary TTP research.

Determine a list of critical systems or segments and any “crown jewel” locations with key organizational stakeholders to understand potential targets.

Discuss the environment-specific details IT teams to better baseline normal behavior. Details might include: privileged account review, network architecture and segmentation discussion, data flow discussion or organizational policy review. The goal should be to tailor hunt analytics to avoid false positives.

3. Hypothesis Formation

Using the outcomes of the intelligence analysis, the hunt team will develop hypotheses targeting the activities threat actors might perform in the environment or how they would operate during each phase of an attack. These hypotheses will be used to guide the behaviors the team looks for during their engagement.

The use of a hypothesis will help to keep the team focused during a hunt and is critical to ensuring the process follows the “structured and repeatable” tenet highlighted above.

As an example, one might begin a hunt utilizing the following hypothesis, “Adversaries use DNS to command and control malicious tools and exfiltrate data from the environment”.

4. Analytic Creation

Based on their hypothesis, the hunt team will develop very specific analytics, or queries, to identify the targeted threat actor behavior. The team’s intelligence and capability with their available toolset will greatly impact the possibility of success. Continued use and learning on each available tool will further the team’s ability to draw relevant and accurate conclusions based on available data.

Using our example hypothesis around DNS command and control, analysts might create several analytics to identify data of interest, such as looking for DNS queries of greater than 150 characters in length, or queries that resolve to non-routable, non-RFC1918 IP addresses.

5. Execution

Using any relevant data, live or historic, the hunt team will execute their test cases and retrieve results for follow on analysis. In practice, the duration for this phase will vary heavily and be dependent upon several factors including scope, data collection, complexity or technologies in place. There is no set time frame best practice to follow as long as your test cases comprehensively explore your hypothesis and your data is sufficient for comprehensive testing.

6. Investigation and Validation

Hunt personnel or another designated team will investigate and validate the identified events to determine whether the identified activity is malicious. This step is where the team’s work in the “Getting Visibility” process will either pay off or identify the need to gather more data or context. During this process, data will be thoroughly analyzed and context will be sought to try to separate true positive detections from data that matches the hypothesis but is not actually malicious. Findings from this phase will lead into the next phase to increase efficiency and detection rates. Additionally, any true positive detections during this phase will likely be transitioned into a proper incident investigation.

7. Revise Hypothesis and Analytic

After execution, the hunt team will document ways in which the analytics or hypothesis can be revised in future hunts to eliminate false positives and improve investigative efficacy.

Due to the manual nature of threat hunting, these programs can be costly, meaning any gains in efficiency will have a magnified return on investment. Hunt teams should always look for ways to improve their speed and efficiency while maintaining accuracy, in turn, reducing the mean time to detection for the greater organization.

Outcomes

The first question people have about threat hunting programs is, “What if they don’t find anything?,” with the implication that it was all a giant waste of time. On the contrary, all hunts should provide measurable value to the organization, even if the end-result was not a security incident. The following are all possible benefits of a hunt not resulting in an incident:

The addition of new high-fidelity indicators for use by the SOC in traditional detection methodologies. Due to the cost of hunting programs, this is the ideal output of any hunt, enabling the automation of future detections.

Security team gains a more in-depth understanding of their environment.

Detection of and identifying issues not originally planned in the hunt (shadow IT, risky behaviors or configurations).

The addition of technology, training of people and creation of process that leads to a significantly improved holistic information security program.

A data-backed measurement of which tactics, techniques and procedures you have cleared in the environment therefore reducing unknown exposure.

Conclusion

Threat hunting is a natural extension of network, endpoint and application level visibility that can lead to meaningful results for security operations teams by providing a more active form of threat detection in your environment.

While developing a hunt program can seem ominous, most organizations are able to start small, leveraging existing data sources in new ways, and eventually expand to incorporate additional threat actor TTPs.

We hope the framework included in this post will combine with other public material to serve as a starting point for organizations to implement their own hunt programs. Those who hunt will inevitably reduce risk in their environment through increased knowledge and security posture or the elimination of threat actors operating under the pre-existing radar.