Understanding the anatomy of a potential incident can be one of the most challenging tasks that an incident response team faces, especially in the increasingly complex, cloud computing environments most organizations have today. But even the most sophisticated hacker can leave behind footprints that can help incident responders piece together what happened to try and prevent a repeat. While using a forensics tool to extract artifacts from endpoint memory is the typically the most comprehensive method of reconstructing a potential incident, it’s also the most time- and resource-intensive. And when responding to a security incident, time is of the essence, particularly with the increasingly stringent data protection requirements set by numerous government regulations and industry standards.
Detecting and containing a security incident is no easy feat in the simplest of network architectures, and the more complex the network, the more difficult detection becomes. Dwell time (the time between initial compromise and detection) can vary from a few hours to several months.
To improve overall data security and minimize the risk of security incidents, organizations need to implement a proactive threat detection plan in addition to a reactive incident response activity. But where to start? It is often like searching for the proverbial needle in a haystack, but certain categories of artifacts can provide the initial insights and can be extremely relevant when performing a live disk analysis of an endpoint.
Generally, the time and resources required to gather and analyze relevant artifacts from multiple machines across an enterprise simply doesn’t scale in large cloud computing environments. To address this scalability problem, the Adobe security team is working on a new approach to digital forensics and incident response (DFIR) that’s quick and cost-effective based on OSQuery – an open source tool that you might already have in your endpoint monitoring toolkit.
The OSQuery framework exposes the operating system of an endpoint as a relational database, against which you can run standard SQL queries to find specific artifacts about the system, such as running processes, logged-in users, open network sockets, bash history, listening ports, process trees, and even Docker containers. Each artifact type gets its own table within the database. Because it uses industry-standard SQL, the query language employed by OSQuery is generic across operating system platforms. While this is a huge plus, don’t mistake it for an alternative to using a real-time response EDR tool.
OSQuery is well-suited to two fundamental DFIR use cases, each of which organizations likely encounter every day. The first use case focuses on forensic data collection for analysis and large-scale threat detection across the enterprise. In this scenario, organizations run OSQuery in interactive mode (osqueryi) from the command line and use ad-hoc SQL queries to collect data during a forensic investigation. For example, you suspect that an endpoint has been infected, but you don’t want to push the button on your forensics tool right away or maybe you’re hesitant to run an artifact script on a compromised endpoint. Instead, you can create a set of queries in OSQuery that are tuned for a forensic investigation and collect the relevant artifacts from that particular endpoint, from broad queries that return all processes running or new services that have been installed as scheduled jobs to more granular searches to detect what may have been loaded by a specific malicious process. Interactive mode is also useful for testing queries before large-scale deployment.
Alternatively, OSQuery is also extremely useful for threat hunting in your enterprise. In this scenario, you install OSQuery as a service (or in daemon mode) and run scheduled queries for periodic data collection. This enables you to determine baseline behavior and then identify outliers that might indicate a potential security threat. Daemon mode, osqueryd, is used for large-scale deployment of queries across your enterprise after you’ve tested them in osqueryi.
The high-level architecture of deployment is shown in the figures below – (1) the osquery agent checks in, (2) the TLS endpoint responds with a query and (3) osquery replies back with the results.
Fleet from Kolide is an Open Source Query Manager through which queries can be deployed via query packs and run across your fleet. The output can be sent to a SIEM or Log Aggregation Platform for further investigation and/or threat hunting.
OSQuery architecture
Using Kolide
Let’s talk about a potential attack scenario involving malicious cryptomining.
The attacker compromises an instance with stolen or default credentials. The attacker also creates a new user account and deletes the legitimate users and blocks legitimate access. At this point, the attacker might spin up more instances. Eventually the attacker installs and starts the Miner. The miner establishes connection to its pool. The attacker might establish persistence and propagate laterally in the network.
Here’s the scenario that we emulated in our sandbox environment.
Sandbox scenario
The detection mechanism in SIEM:
Example in SIEM
While I’ve only touched on its capabilities at a high level, OSQuery can be an ideal solution to help incident response teams to work quickly and at necessary scale when a security incident occurs. Even the stealthiest attacker can leave behind footprints—and the quicker and more accurately you can detect and analyze these footprints, the faster you can determine the scope of the threat and work to contain it.
For more information on Adobe’s scalable approach to DFIR using OSQuery, watch this webinar.
Sohini Mukherjee
Security Analyst, Adobe Security Coordination Center (SCC)
Andres Martinson
Sr. Security Engineer, Experience Cloud