Craftmans Guide to Cybersecurity Investigations Part 1

The Craftsman’s Guide to Cybersecurity Investigations – Part I: Finding a Tree

In the critical cybersecurity arena of incident response, there has been a significant lack of progress on improving dwell time, mean time to detect/discover (MTTD), and mean time to respond/remediate (MTTR). A key reason that these areas continue to be a challenge is that security investigations remain complicated, manual, and slow due to the common yet archaic approach of universal data centralization. Once sufficient, this approach is no longer adequate with data proliferating across so many systems and platforms that reside in the cloud, third-party SaaS, and on-prem environments.

Being a cybersecurity analyst today is like being a craftsman looking to make the perfect dining room table. First, they need to understand the characteristics of the thousands of species of wood available to them around the world. Then, they need to hunt across the millions of acres of forestry available to find just the right trees with the right characteristics to mill. And because they are expert craftsmen, they know the perfect table doesn’t consist of just one wood but multiple species each adding their own bit of character to the piece. Talk about a daunting task, especially when you may have dozens of more orders waiting to be filled.

Don’t see the correlation? Let’s look deeper into the process of a cybersecurity investigation to make it more obvious.

Similar to that dining room table for a craftsman, a security investigation usually starts with a triggering event such as an alert, a third-party notification, or threat hunting (the table order). From there, the arduous journey begins with security analysts using their knowledge of adversarial behavior and the typical business activity in their environment to begin a search for all relevant data (trees). This entails pivoting between dozens of browser tabs to search all sorts of data silos, including security information and event management (SIEM) systems, other data lakes and repositories, endpoint detection and response (EDR) technologies, identity and access management (IAM) solutions, threat intelligence, and the list goes on and on. Data types found during the investigation might include endpoint telemetry, emails, web proxy logs, IP addresses, user names, file info, and countless others (the tree’s character). Phew. That’s exhausting just to read.

So, let’s get into the heart of why this process is such a problem for security operations teams:

  1. There is a steep learning curve to become a security analyst, as understanding the multitude of unique security tools is an essential skill. Add to that some new data from U.S. News & World Report, which ranks a security analyst as the #1 job in 2022 and predicts more than 47,000 new jobs in this field opening up by 2030. It’s going to become even more difficult to recruit and retain these individuals. So, in many cases the person investigating an event will not have the required skills to access the data needed for an investigation and will need help from another overburdened analyst to get the job done. Cue the revolving door of your security analyst team.
  2. Even for experienced analysts, the current process is a wildly inefficient, “artisan” approach to investigation: asking the same questions in multiple languages to different tools and systems. There has to be a better way, don’t you think?
  3. The universal “centralization” approach to security data has fallen short of its vision, and it has created more noise and systems to manage: SIEMs, data lakes, and now the newest remake of the same movie, extended detection and response (XDR). The security alphabet soup continues to expand. In my 20+ years of experience I have yet to see a single customer who has all their data and required context in one platform. It’s impossible.

The result of this antiquated approach? Each security investigation gets bogged down before it even gets started. Imagine if that craftsman had a huge team searching for that perfect tree for him so he could work on the dining room table and move onto those dozens of other projects more quickly. Wouldn’t that be nice? In my next post in this series, The Craftsman’s Guide to Cybersecurity Investigations – Part II: Examining the Tree, I will share more insight into what is typically required to carry out an investigation today and why it matters.