A Lesson From the CrowdStrike Incident

COMMENTARY

In the world of cybersecurity, no company is immune to outages and breaches. Recently, CrowdStrike, a leader in endpoint security, experienced an outage that affected many of its users. While the details of the root cause analysis are still emerging, I suspect there's a strong likelihood that the issue stemmed from a breakdown in process adherence — specifically, code that might not have gone through the proper testing, security reviews, or governance before deployment.

Why Process Matters

The output of any business process is predicted by adherence to the process. Put another way, a process is never going to be predictable (or maximally efficient) if it isn't followed. That sounds like a tautology, but there is a misguided train of thought that somehow software development is the exception to this rule.

These processes include key steps like code reviews, security checks, and user acceptance testing. When well-meaning people ignore the process, usually trying to work quickly, the risk of errors and vulnerabilities skyrockets.

We often talk to technical executives who confess that they have very little in the way of process controls. Chief information security officers (CISOs) who are unaware that key code changes didn't go through security review or chief information officers (CIOs) whose teams aren't aware of architecture changes.

Even worse, when executives believe their teams mostly follow their process, we often find that a quick audit of the systems of record shows that to not be the case.

If our suspicions prove correct, the CrowdStrike outage serves as a reminder that even the most advanced and well-respected companies can experience failures when process adherence falters. In many cases, these failures can be traced back to a lack of visibility and accountability in the process itself.

The pernicious thing about process is that it can seem like a tax on productivity until it's too late. In our financialized world, the risk of outage or breach isn't held on a balance sheet anywhere. Technical debt is real but isn't tracked by the Financial Accounting Standards Board or generally accepted accounting principles.

Over time, improving processes leads to greater efficiency and productivity, but the change management can be hard, and it can be especially difficult to know where to begin and what process change to prioritize.

How to Avoid Future Incidents

Ensuring process adherence and visibility can significantly enhance an organization's security, reliability, and overall performance, thereby avoiding costly outages and maintaining the trust of their customers. 

Key Steps to Ensure Process Adherence

1. End-to-end process visibility: Comprehensive visibility into every step of business processes is crucial. This transparency ensures that all activities, from code development to deployment, are monitored and recorded, making it easier to identify and address potential issues before they escalate.

2. Automated compliance checks: Automating compliance checks to ensure that all necessary reviews and approvals are completed before any code is shipped reduces the likelihood of human error and ensures that all code meets the required security and governance standards.

3. Real-time auditing and reporting: Real-time auditing and reporting capabilities allow organizations to continuously monitor their processes and generate detailed reports. This functionality helps in quickly identifying deviations from the established process and taking corrective actions promptly.

4. Proactive risk management: Advanced analytics and risk management tools can help organizations proactively identify and mitigate risks. By analyzing historical data and trends, potential issues can be predicted, and preventive measures can be suggested, thereby minimizing the impact of any disruptions.

Conclusion

The recent outage at CrowdStrike highlights the critical importance of adhering to established processes and governance frameworks. As we continue to navigate the complexities of the digital landscape, remember that robust processes and effective governance are the cornerstones of success. By focusing on process adherence and visibility, organizations can avoid future incidents like the CrowdStrike outage, ensuring their business remains resilient and secure.