The True State of DevSecOps

As enterprises increasingly unite security principles and standards within DevOps practices, they're speeding up software delivery and improving security in the process.

A new tide of research and anecdotal evidence indicates that the marriage of the two, known as DevSecOps, is helping them get better at automating security testing and improving security attributes of applications earlier in the development process.

But at the same time, the data and war stories show that there's still a lot of work to go before DevSecOps work patterns fully mature within the enterprise. At most organizations, it's still a struggle to fold security tools into the DevOps-optimized software delivery pipeline. And the majority of DevOps stakeholders still see security as an inhibitor to DevOps agility.

The most recent research came by way of Sonatype this week with its DevSecOps Community Survey. On the good news front, the study shows that in the past three years, the ratio of organizations that test applications throughout the development lifecycle compared to just in production has grown significantly, with close to 2x gains at nearly every early stage step in the development process.

Those organizations that test or analyze for security requirements throughout the entire development process increased from 15% of organizations in 2013 to 27% today. What's more, among those organizations with high DevOps maturity practices, 42% reported that they test throughout the lifecycle. That higher rate is likely influenced by higher rates of automation: 58% of highly mature DevOps shops automate security testing compared to just 39% of other organizations.

Nevertheless, security still suffers from a perception problem. The survey showed that 59% of organizations believe security is an inhibitor to DevOps agility. A big part of the difficulty is that many security testing tools today are still too far removed from the typical developer's workflow and tool chain.

While the DevSecOps ideal is to embed security testing directly into the software delivery process, actually doing it is "a whole other ball game," said Adam Jacob, CTO of Chef, an IT workflow automation vendor that plays heavily in the DevOps world, in a recent podcast.

"If you can't figure out how to manage that security posture the same way you manage the rest of what you do, it's really difficult to then tell a software developer that it’s their responsibility to ensure that that posture is good or bad," he explains. "You can't really ask them to understand the posture of what it's going to be like when it's deployed.

"Because the distance from a software developer making a decision to a software developer talking about how that software should be in production - and what its posture ought to be - is so vast. And their ability to influence it is so low."

Of course, security professionals struggle to bring these tools closer to the developer because the majority of security testing tools were designed for traditional waterfall-development models.

"For those of us who have been involved on the front lines of traditional AppSec activities such as penetration testing, dynamic- or static-code analysis, it may be obvious that the traditional tools and techniques we use were built more for waterfall-native rather than DevOps-native environments," says Oleg Gryb, chief security architect for Visa. "Yet for executives who came to security from infrastructure, networking, or development domains and have never run a security scan, the challenges of bringing traditional tool sets and practices into the new velocity expectations of DevSecOps may not be so obvious."

This causes big hiccups in testing, such as in the case of compliance validation. A survey published last week from Chef reports that 64% of DevOps shops have regulatory standards to follow. Of those, 73% wait to assess compliance after development, and 59% wait all the way until code is already running in production. This results in a lot of added strain on developers and inconsistent security, to boot.

The Chef survey showed that after a compliance violation is found, one in four organizations need weeks or months to remediate them. In a DevOps world where dozens or even hundreds of builds a day are the delivery norm, this is a positive geologic age in time progression before fixes are made.

According to DJ Schleen, security architect at Aetna, this will require security leaders to not only look for better tools but also get creative about where and how security controls are automated into the workflow.

[Chef executives will present Defining DevOps Metrics at Interop ITX on Tuesday, May16, at the MGM Grand in Las Vegas.]

"We need to identify ways to observe and collect data in both a passive and an active way. Passive ways of producing data could include methods to calculate defect density for a collection of code after a static code analysis has been performed, or to calculate the risk ranking for a codebase based on code smell - whereas an active way of producing data may be the action of pausing the release pipeline to perform a vulnerability scan itself," he says.

Schleen says active collection is tricky when integrating automated tools."When you are generating data in an active way, you can potentially be a bottleneck in your release pipeline," he notes.

Additionally, security folk would also do well to challenge their comfort zone with regard to tools, says Troy Marshall, DevSecOps and cloud reliability leader for Ellucian, a developer of software for higher education. That's exactly what his firm had to do when it found that the commercial dynamic application security testing (DAST) tool it was using prior to its DevOps transformation wouldn't fit well into its new software delivery tool chain.

"We looked at a lot of other commercial tools and decided that the available open source tooling was sufficient for our minimum viable product," he says, explaining that there have been bumps along the way but his team has made it work.

Related Content: