Citrix ADC, Gateways Still Backdoored, Even After Being Patched

Nearly 1,900 Citrix networking products around the world have been backdoored as part of a large-scale automated campaign targeting CVE-2023-3519, according to researchers from Fox-IT, part of NCC Group. The adversary appears to have exploited and placed Web shells on vulnerable Citrix NetScaler Application Delivery Controllers (ADC) and Citrix NetScaler Gateways to establish persistence.

The presence of the Web shell means the adversary can continue to remotely execute arbitrary commands, even after the appliance has been updated and/or rebooted. Last month, when the vulnerability was first disclosed and update released, Mandiant researchers identified over a half-dozen Web shells that attackers are using to modify the NetScaler configuration and stop or deactivate processes and services.

"Notably since at least 2021, cyber espionage threat actors have focused on edge devices, particularly security, networking, and virtualization technologies, to gain persistent access to victim networks while evading detection," Mandiant researchers said at the time.

At the time of this automated campaign, which Fox-IT researchers estimate took place between July 20 and July 21, 31,127 NetScalers were vulnerable to this remote code execution flaw. As of Aug. 14, 1,828 NetScalers have some kind of backdoor. Of the compromised systems, 1,248 of them have already been patched for this vulnerability.

"While most administrators were aware of the vulnerability and have since patched their NetScalers to a non-vulnerable version, they have not been [properly] checked for signs of successful exploitation," Fox-IT said.

This is one of those times when just updating and rebooting the system is not sufficient. Enterprise defenders should still check their NetScalers for clues they have been compromised, regardless of when the patch was applied. Mandiant's IoC Scanner is a bash script checking for indicators of compromise on NetScaler appliances.

If a Web shell is found, defenders can look for signs in the NetScaler access logs to see whether the Web shell was used. If those clues are present, then further investigation is necessary to identify whether the adversary has moved laterally through the network, Fox-IT said.

The majority of compromised NetScalers appear to be in Europe. While Canada, Russia, and the US all had thousands of vulnerable NetScalers on July 21, virtually none had Web shells installed, Fox-IT said.

Mandiant's researchers noted that research in previous operations, including against these same appliances last year, shows that this campaign is consistent with activities by espionage threat actors linked to China. Citrix had already reported and patched a similar vulnerability in its ADC and gateway appliances that was being actively exploited back in December 2022.

"Mandiant has investigated dozens of intrusions at defense industrial base (DIB), government, technology, and telecommunications organizations over the years where suspected China-nexus groups have exploited zero-day vulnerabilities and deployed custom malware to steal user credentials and maintain long-term access to the victim environments," the researchers said.