P.F. Chang's Breach Went Undetected For Months

On June 13, restaurateur P.F. Chang’s notified customers via email of a security breach, stating credit and debit card data had been stolen and the company only recently became aware of the breach -- on June 10, 2014. According to a dedicated website for communicating with the public, P.F. Chang’s officials said they learned of the breach from the US Secret Service and are currently working with PCI Forensics Investigators (PFI) to determine its scope.

Although it will take time for investigators to complete the investigation, Brian Krebs reports having a copy of a seemingly related alert sent by the card brands to banks indicating the compromise goes back at least as far as September 18, 2013. These alerts are sent by card brands as they receive fraud reports and look through transaction history to identify a common point of purchase (CPP) from which the cards were likely to have been stolen.

Additionally, Krebs notes that all P.F. Chang’s locations began using old-fashioned “knuckle busters” where a physical imprint of the card is made instead of relying on electronic payment processing. In my experience handling merchant breaches, switching to knuckle busters or standalone terminals where they’re not connected to the point of sale (POS) network is standard practice to continue credit card acceptance without disturbing the POS environment.

Between the information provided by P.F. Chang’s and the information received by Krebs, it’s likely that most, if not all, locations are affected. P.F. Chang’s is privately owned, which means it has control over the IT infrastructure at all locations. That makes this breach very similar to those at Neiman Marcus and Target in terms of potential scope. In all likelihood, a large percentage of its locations were indeed affected.

Who’s to blame?
First and foremost, let me make it clear that the criminals are to blame. P.F. Chang’s is a victim. Based on experience dealing with card data breaches of this magnitude, it’s likely that P.F. Chang’s or a third-party IT services provider overlooked one or more security controls when hardening its network. There could even be a shared responsibility between P.F. Chang’s and an IT services provider, depending on the dynamic of that relationship.

As the intrusion was in progress, it’s also possible that alerts went unnoticed or dismissed as understaffed and underequipped security teams were forced to pick and choose which alerts warranted action. Only in hindsight during the investigation will those alerts stand out. Finally, it could come to light that a seemingly unrelated third party was involved, as was the case in the Target breach where its HVAC service provider was compromised and used to gain initial access.

When seeking to point fingers, keep in mind that complex systems fail usually as a result of multiple points of failure. It’s a fool’s errand to seek out a single cause. Even if an initial entry point were better secured or alerts were investigated, determined and skilled attackers are smart enough to dodge and weave their way to their ultimate goal.

How did this breach unfold?
The public will never know. At best, one or two points of failure will be selectively disclosed as scapegoats. Speaking from first-hand experience, I can provide general information on the attack lifecycle true of most incidents.

The attackers first find an initial point of entry into the victim network such as SQL injection against a web server, remote access using stolen employee credentials, or backdoor access through common droppers or backdoors that made their way to employee systems. Those backdoors could have been deliberately planted by the attackers, or they could be from different opportunistic attackers that sell access on the black market. From that starting point, attackers focus on privilege escalation to get administrative access, lateral movement using built-in system commands to map out the environment, and additional backdoors to ensure access in the event they get busted. Then they locate pivot points into the card data environment (CDE).

Once in the CDE, attackers harvest card data for a long time using specially crafted malware that snags it from memory, LAN traffic, or through keystroke recording. Sometimes they can simply compromise encryption keys and query POS databases. The final step is periodically transmitting the stolen data out undetected. Common methods involve creating an encrypted archive, which bypasses systems designed to identify card data, then sending it out in small chunks to avoid systems monitoring for spikes in network traffic.

What are the lessons learned?
My advice to consumers and businesses continues to be the same. Consumers should monitor financial accounts for unauthorized transactions so they can report them quickly and avoid being held liable. They should prefer credit over other payment options because of laws protecting them from losses. My advice to businesses concerned with security incidents of any kind is to engage security staff in a discussion focused on risk. That will provide the data points needed to make an informed decision on budgeting and prioritization.

With budget and authority in hand, security teams should seek out ways to more effectively manage existing security investments and reduce the noise. Common approaches are fine-tuning current products and automating as much as possible. Next, they should look to improve rapid detection and response capabilities to stay on top of things and ensure that attacks are identified and thwarted before damage is done.