SCAHunter: Scalable Threat Hunting Through Decentralized Hierarchical Monitoring Agent Architecture

Abstract

This paper presents a scalable, dynamic, flexible, and non-intrusive monitoring architecture for threat hunting. The agent architecture detects attack techniques at the agent level, classifies composite and primitive events, and disseminates seen attack techniques or subscribed event information to the upper-level agent or manager. The proposed solution offers improvement over existing approaches for threat hunting by supporting hierarchical event filtering-based monitoring, which improves monitoring scalability. It reduces memory requirement and communication overhead while maintaining the same accuracy of threat hunting in state-of-the-art centralized approaches. We provide a distributed hierarchical agent architecture and an approximation algorithm for near-optimal agent hierarchy generation. We also evaluated the proposed system across three simulated attack use cases built using the MITRE ATT &CK framework and DARPA OpTC attack dataset. The evaluation shows that our proposed approach reduces communication overhead by 43% to 64% and memory usage by 45% to 60% compared with centralized threat hunting approaches.

Appendix A

In order to build the attack signature in terms of low-level system events, we follow the technique detectors provided by LogRythm [5]. Following the rules developed by LogRythm, technique T1189 can be detected by inspecting IDS or antivirus logs. Technique T1035, T1021.001, and T1119 [1] can be detected by analyzing Windows event logs or SysMon logs as mentioned by MITRE [1]. Moreover, technique T1048 can be detected by analyzing network traffic and looking for uncommon data flow in the NetMonitor. Therefore, we can define the signature in terms of low-level logs for technique T1189, T1035, T1021.001, T1119, and T1048 [1], and use case 2 using ESP formalization provided in Sect. 3 as follows:

$$\begin{aligned}&IDSMon.\_event\_id == Malware\_detected \wedge \nonumber \\&Malware\_detected.malwareFileDir == '*temp' \wedge \nonumber \\&SysMon.\_event\_id == object\_access \wedge \nonumber \\&object\_access.object\_dir == '*temp' \wedge \nonumber \\&object\_access.procName = 'chrome' \end{aligned}$$

(10)

$$\begin{aligned}&SysMon.\_event\_id == service\_creation \wedge \nonumber \\&service\_creation.command == 'SC\ create' \wedge \nonumber \\&service\_creation.imagepath == '*temp' \end{aligned}$$

(11)

$$\begin{aligned} SysMon.\_event\_id == RDS\_logon\_success \end{aligned}$$

(12)

$$\begin{aligned}&SysMon.\_event\_id == process\_created \wedge \nonumber \\&process\_created.command == '*.bat' \wedge \nonumber \\&SysMon.\_event\_id == network\_conn\_created \end{aligned}$$

(13)

$$\begin{aligned} NetMon.\_event\_id == uncommon\_data\_flow \end{aligned}$$

(14)

$$\begin{aligned}&(Equation~10) \wedge (Equation~11) \wedge (Equation~12) \wedge (Equation~13) \wedge (Equation~14) \wedge \nonumber \\&(object\_access .object\_dir == service\_creation.imagePath ==\nonumber \\&malware\_detected.malwareFileDir) \wedge (RDS\_logon\_success.src\_ip ==\nonumber \\&ids\_event\_id.host\_ip) \wedge (RDS\_logon\_success.session\_id == \nonumber \\&process\_created.session\_id == network\_conn\_created.session\_id) \wedge \nonumber \\&(uncommon\_data\_flow.network\_protocol == \nonumber \\&network\_conn\_created.network\_protocol ) \end{aligned}$$

(15)

To build the attack signature of the red team activities in OpTC attack dataset, we investigate notepad++ update process and Meterpreter execution process. To detect Meterpreter payload download activities, we can look for new file create and write event logs and process creation using the newly created file.

$$\begin{aligned} a.Operation == NewFileWrite \wedge x.event\_id == process\_creation\nonumber \\ \wedge x.imagePath == a.newFilePath \end{aligned}$$

(16)

Named pipe impersonation can be detected by looking for any cmd.exe process which is a child of services.exe and the commandline arguments of the cmd.exe process contains echo and pipe keywords.

$$\begin{aligned} b.processName== cmd.exe \wedge b.parentProcess== services.exe\nonumber \\ \wedge b.Commandline \in echo \wedge b.Commandline \in pipe\nonumber \\ \wedge b.process\_id == x.process\_id \end{aligned}$$

(17)

The system info, installed applications, domain controllers and network share discovery activities can be detected by looking for command line arguments running form the cmd.exe process which is spawn from Meterpreter process.

$$\begin{aligned}&(c.commandline \in [local\_system\_info\_enumeration\_command \nonumber \\&\cup installed\_application\_enumeration\_command (tasklist) \nonumber \\&\cup domain\_controller\_enumeration\_command (dclist) \nonumber \\&\cup network\_share\_enumeration\_command (Get\_SmbShare)] \nonumber \\&\wedge c.process\_id == b.process\_id) \end{aligned}$$

(18)

We can detect the registry key creation and setting the value of the created key to a newly downloaded payload as follows:

$$\begin{aligned}&g.Operation == NewFileWrite \wedge h.Operation ==&RegistryKey\_create \nonumber \\&\wedge g.newFilePath == h.registryKeyCreated.value \nonumber \\&\wedge g.process\_id == h.process\_id == x.process\_id \end{aligned}$$

(19)

Creating new user account and adding it to specific group can be done by using net utility. Thus the signature will be:

$$\begin{aligned}&i.Commandline \in net\_userAccount\_add\_command \nonumber \\&\wedge i.process\_id == b.process\_id \wedge i.processName == Net.exe \end{aligned}$$

(20)