5 Key Questions When Evaluating Software Supply Chain Security

Software supply chain compromises like the ones SolarWinds disclosed last December and Kaseya in July have become a growing threat. Indeed, cybercriminals appear to have glommed onto the breach-once, infect-many attack model and are ramping up attempts to break into software companies.

The trend has raised so much concern that US President Joe Biden made software supply chain security a key priority in a cybersecurity executive order he signed in May. The order requires all civilian federal agencies to take measures to evaluate and verify the security practices of their suppliers. It also mandates new guidance to be developed that will eventually require software developers to maintain secure development environments, implement strong controls for accessing their network, use encryption, provide purchasers with a software bill of materials for each product and a slew of other measures.

In addition, the Department of Homeland Security's US Cybersecurity and Infrastructure Security Agency (CISA) and the National Institute for Standards and Technology (NIST) have developed a new resource designed to give federal agencies an overview of software supply chain risks and measures for defending against them.

So what do organizations need to be thinking about when evaluating the security of their software supply chains and their suppliers? Here, according to security experts, are five questions worth asking.

Have You Pen-Tested the Software?
Asking the right questions can provide a lot of information about a software supplier. But questions alone are not enough. Where possible, security managers need to do their own due diligence. Penetration tests are one way to do this.

"Penetration tests can provide valuable insight if an application's business logic and security controls — [such as] authentication and authorization — can be bypassed," says Nasser Fattah, chairperson of the North America Steering Committee at Shared Assessments.

Software vendors are not going to provide their source code for review because it's intellectual property, he adds, so it's important for organizations to fully understand the expected behavior of their software so they can monitor for unexpected behaviors.

Where possible, says Jack Mannino, CEO at nVisium, security organizations should consider analyzing the code itself, using a third party if needed. "In many cases, software is backdoored using techniques that evade simple detection," Mannino says. 

Exploring as many code paths through the software as possible, following control flows, and understanding how the software behaves are keys to unearthing potentially hidden security issues in the software, he adds.

How Does the Vendor Assess Security of Its Software?
Companies purchasing and installing vendor software in their environments need to know the vendor's software security protocols, Fattah says. For instance, does the vendor have an integrity process that checks for software updates? How often has the software been updated? Does it run in least-privileged mode by default? 

"Many software [products] require [running] in privileged mode, which means the software can take full possession of the system that it is running on," he says. Examples of products that require high privileges include security tools and network management tools, such as SolarWinds' Orion platform.

Also, understanding your software's bill of materials — or what comprises the software, including open source components — is key to knowing what vulnerabilities to monitor for, Fattah says.

And according to Hank Schless, senior manager, security solutions at Lookout, having full visibility into a vendor’s security practices "is one of the most important parts of the vetting process. You need to understand what controls they have in place, how they monitor access to your data, and what they have to protect your data in the case of a security incident on their end," he says. 

Have your IT, engineering, and security teams take a look at any third-party software you're onboarding, he advises. "The new vendor should be open to this process," Schless adds. "If they’re hesitant, that could be a red flag."

How Is the Vendor Checking for Vulnerabilities in Its Software Product?
Verify what measures the software vendor has in place for checking its product for known vulnerabilities. One important question to ask is what commercial or open source application testing tool or service it uses to assure its software is free of known security defects and malicious capabilities, says John Pescatore, director of emerging security trends. Check whether it can provide a copy of its last test run, he says.

"A companion question is, have they done any benchmarking/maturity level assessment of their software development processes and can you see that?" Pescatore says. If the vendor answers no to both questions, it's best to avoid it unless the situation involves a must-have technology, Pescatore says. 

"Then its use should be as segmented as possible," he says, adding that all admin accounts should require two-factor authentication. There also needs to be continuous monitoring of logs and network traffic from and to that software, he adds.

How Is the Vendor Securing Its Network and Devices?
Attackers like targeting the software supply chain because by compromising one vendor, they have an opportunity to compromise numerous other companies. So it's important to know how the vendor manages and secures access to its digital assets.

Organizations should consider using the services of third-party cybersecurity rating firms to get an idea of a vendor's security posture, security experts say. Such firms calculate security ratings — similar to credit scores — for organizations based on data gathered externally about the company from the Internet. This might include communications to and from compromised systems, file sharing, signs of brute-force attack, and Internet of Things (IoT) traffic. 

The data can often reveal how vulnerable an organization might be via open ports, software and network vulnerabilities, configuration errors that are visible externally, and a range of other security issues. It also highlights issues like the absence of web application firewalls, unpatched systems, and the use of obsolete SSL protocols.

A.J. King, CISO at BreachQuest, says using these services can provide organizations with a quick idea about the security practices and maturity of the vendor they are considering. Does the vendor have low scores in patch management? Has it maintained and published email security protocols, such as SPF, DMARC, and DKIM? 

"These are indicators of how important they view security as a company and will help you measure the overall risk of the firm," King says. 

The services also have continuous monitoring features that alert security teams when a vendor it's using has experienced a significant change in its security posture, he notes. Other questions to ask include when the app vendor conducted a full web application penetration test, whether it has a bug bounty program, and what its policies are for engaging with ethical security researchers who might find flaws in its software, King says.

Joseph Carson, chief security scientist and advisory CISO at ThycoticCentrify, says security teams also need to find out how the vendor protects and verifies privileged access to crucial data and systems. 

"How do they maintain an audit log of activity, how do they encrypt sensitive data, and what risk is exposed to your organization if they are ever compromised?" he says.

Does Your SaaS Vendor Align With the Organization's Risk Appetite?
All the questions that organizations ask a software provider before onboarding its software on-premises are relevant when dealing with a software-as-a-service (SaaS) provider, too. But there are a few additional issues organizations need to make sure they understand when considering a SaaS provider.

For example, it pays to find out whether customer data leaves the vendor's production system, says Demi Ben-Ari, CTO and co-founder of Panorays. Does the vendor support single sign-on for internal systems or customer access, and does it comply with regulations that are important to your company, such as the European Union’s General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), and the Payment Card Industry Data Security Standard (PCI DSS)?

Also verify whether the SaaS provider has processes for ensuring that employees access customer data only on a need-to-know basis. Find out how it secures internal Wi-F networks and how it has hardened critical servers against attack. 

"Does the vendor have controls in place to prevent unauthorized access to its applications, program, or object source code?" Ben-Ari says. "Does the vendor encrypt data stored in the cloud?