Living With Compromised Technology Supply Chains in a Post-Supermicro World

The reactions to Bloomberg's controversial scoop about Chinese spy chips embedded into the hardware of major server suppliers is shedding light on the difficulties enterprises and governments face in contending with a global technology supply chain that is growing ever more complex and convoluted.

The gist of the October 4 story by Bloomberg is this: Chinese-government affiliates compromised (or ensured the compromise of) the manufacture of select motherboards in China by having a tiny chip embedded within the hardware itself, which was capable of sucking out and transmitting data. (See: China Hacks Hardware in Spying Attempt on Apple, Amazon & Others – Report.)

These motherboards, being manufactured by and for hardware firm Supermicro, would ultimately wind up in machines being used within major government and financial enterprises in the US.

The Bloomberg report also implicated major companies such as Amazon and Apple in discovering the flaw and quietly working to mitigate their losses afterwards -- allegations that both companies vehemently deny. Other pundits have raised their own doubts based on Bloomberg's compensation scheme, which rewards reporters for "moving markets" with their stories. A recent eWeek write-up went so far as to discount the Bloomberg report by arguing that, because there are easier and more secure ways of compromising motherboards, the Bloomberg report is unlikely to be true. (This is rather like saying that the Watergate break-in didn't happen because the burglars simply could have paid off Frank Wills.)

(Source: iStock)

Bloomberg's reporting, however, clarifies precisely why it was beneficial to plant a spy chip in the way described: ""[Amazon's] security team determined that it would be difficult to quietly remove the equipment and that, even if they could devise a way, doing so would alert the attackers that the chips had been found, according to a person familiar with the company's probe... [There is] no commercially viable way to detect attacks like the one on Supermicro's motherboards has emerged -- or has looked likely to emerge. Few companies have the resources of Apple and Amazon, and it took some luck even for them to spot the problem.""

"Some luck" understates the case. Reportedly, Amazon only discovered these spy chips because of an extensive due-diligence process that involved comparing actual server motherboards from a company Amazon was acquiring against those motherboards' original blueprints.

Considering such luck, how can an enterprise IT organization without the resources of Amazon or Apple discover if its own hardware has been so compromised?

How to catch a spy chip
Hardware-security testing sometimes requires extremely specialized and sensitive equipment (particularly in the case of reverse engineering). It may also depend upon the particular hardware-embedded vulnerability not being designed to mitigate for common testing methods by "faking" outputs and results.

Consequently, most IT-security departments default to rudimentary searches for unusual network activity -- but that can be difficult to diagnose. In the instant case, unusual operating-system activity would have been yet more beneficial to seek out; Bloomberg reported that the microchips embedded on Supermicro motherboards "altered the operating system's core so it could accept modifications."

Either way, however, it would have taken substantial investigative resources to uncover the underlying (and unlikely) problem in a world where IT workers are already overwhelmed by security alerts. (See: Endpoint Security: 3 Big Obstacles to Overcome.)

Solving the macro-problem
The private sector's diagnostic difficulties aside, some suggest that the government should further insinuate itself into solving supply-chain compromises of hardware.

"Hardware interdiction as a means to enable spying or sabotage is a fairly old concept," Ross Rustici, Senior Director of Intelligence Services at Cybereason, wrote in an email. "This incident should force government to re-examine how they inspect and certify critical hardware."

"If there is demand, end-to-end production for critical verticals could shift to the US, and the government could in fact mandate US-sourced hardware," Mukul Kumar, CISO at Cavirin, told Security Now. "But where does it end? Disk arrays? Routers? Switches? Power supplies?"

At a certain point, nationalism becomes the favored guideline -- just as when European firms pooh-poohed US vendors after Edward Snowden leaked sensitive documents detailing high-tech NSA spycraft. But it may not end there. Bloomberg reporters note that "Mandarin is the preferred language" in meetings, whiteboarding, and other communications at Supermicro because most of its employees are of Taiwanese or mainland-Chinese ethnicity; accordingly, Bloomberg avers, these factors may have made California-based Supermicro a target while English-preferring manufacturers with a minority-Asian workforce stayed uncompromised.

"These overseas ties, especially the widespread use of Mandarin, would have made it easier for China to gain an understanding of Supermicro's operations and potentially to infiltrate the company," reads the report.

So goes the supply-chain tragedy for Supermicro -- which, despite being the maker of the compromised motherboards, appears to be an innocent victim of an attack by a nation-state -- and that's good enough (or bad enough) to sow doubt in the market.

"I worked for Supermicro 20 years ago, and I'm sure they didn't have anything to do with the modification of their motherboards," wrote fellow Security Now contributor Pablo Valerio in an email. "But the company is taking a big hit on their stock now. Probably many of their customers are going to start inspecting all their servers."

To be sure, Supermicro stock (NASDAQ: SMCI) quickly plummeted by more than 40% on the news; Amazon (NASDAQ: AMZN) is down close to 10% since then and even Apple's stock (NASDAQ: AAPL) has hit a modest bump.

Unsurprisingly, manufacturers and vendors are judged by the company that they keep -- even if that company is unwelcome.

Related posts:

— Joe Stanganelli, principal of Beacon Hill Law, is a Boston-based attorney, corporate-communications and data-privacy consultant, writer, and speaker. Follow him on Twitter at @JoeStanganelli.