Let's Play! Raising the Stakes for Threat Modeling With Card Games

One Friday evening last month, three security experts met online to play cards and talk about the future of threat modeling. The games they played, OWASP Cornucopia and Elevation of Privilege (EoP), are meant to help developers, architects, and security experts to examine potential risks and rank them according to importance.

Helping organizations identify their threat model has become challenging since the pandemic hit and teams started to work remotely. Online meetings to discuss security were not effective enough and not fun, say Toby Irvine and Grant Ongers, founders of Secure Delivery. They put together an online version of OWASP Cornucopia to address that and invited leading expert Adam Shostack to join them for a game. Shostack created Elevation of Privilege back in 2010 while at Microsoft and used it to help teams identify potential security risks.

"We had this problem of how do we scale security out to everybody," Shostack says. "And the first answer I had was a software tool. And the software tool, honestly, was tedious. So long story short, I asked: What is the opposite of tedious? It's fun. And what is fun? Games. Let's make it a game."

Both Cornucopia and Elevation of Privilege provide developers with "the right context, the right tools, and the right space to think about the problem," all of which makes threat modeling "much more likely to happen," Ongers says.

Irvine agrees. "We can't treat each other as just automatons," he says,

The players are ready — the game, and the philosophical debate around threat modeling, can begin.

Putting the Cards on the Table
At the click of a mouse key, Irvine starts the online version of Cornucopia, which is still in beta. He puts the link inside the video call for Ongers and Shostack to access. The deck has six types of cards: Data validation and encoding, Authentication, Session management, Authorization, Cryptography, and Cornucopia — a category created to accommodate all those cards that didn't fit anywhere else. Similar to poker, each suit has 13 cards (Ace, 2-10, Jack, Queen, and King), and there are also two Jokers: Alice and Bob.

"The A Joker, Alice, is all about how attackers can abuse your system to attack your users," Ongers says. "And the B Joker, Bob, is all about the regulatory issues. The GDPR makes that an easier attack now."

That Friday evening, the threat modeling exercise poked at the online version of Cornucopia Irvine built, so he was allowed to play the first card. He chose the 6 of Session Management: "Gary can take over a user's session because there is a long or no inactivity timeout, or a long or no overall session time limit, or the same session can be used from more than one device/location."

Ongers follows with the 8 of Session Management: "Matt can abuse long sessions because the application does not require periodic re-authentication to check if privileges have changed."

Lastly, Shostack picks the 5 of the same suit, which reads: "John can predict or guess session identifiers because they are not changed when the user's role alters (e.g. pre and post-authentication) and when switching between non-encrypted and encrypted communications, or are not sufficiently long and random, or are not changed periodically."

Each time someone plays a card, they need to read the security requirements and make notes about it. In the end, everyone votes for the best card of the round, and the winner gets a point.

"One of the interesting things that always comes up when I'm training folks is: Why are we doing this?" Shostack says. One answer is that developers and architects constantly make decisions, often without realizing it.

"One of the goals is that those developers know that they are making the choice that they just made," he adds. "Hey, Matt can do this. And Gary can do that. Are we really OK with that?"

Ongers adds: "We approach threat modeling from the point of view of people who are going to be building the thing. [They] should be thinking about how that thing can go wrong. Threat modeling is understanding how applications fit together, how data flows through applications."

Shostack advocates for his card, the 5 of Session Management, explaining how bad it would be if the attacker, John, could predict session identifiers and anonymously connect to the game session. He suggests applying more security to prevent that.

"Maybe I should have had to be logged into my accounts before I get into it," he says. "Do I get a point?"

The issue is of serious concern, so Irvine and Ongers vote for it, and Shostack gets his point.

"He's currently playing the highest card around," Irvine says. "The man is on fire!"

Showing Your Hand
Card games like Cornucopia and Elevation of Privilege allow tech professionals to do threat modeling in a healthy way, Shostack says. "One thing people get wrong is trying to think like an attacker or make it about mindset rather than technique," he adds. "The cards are all intended, in various ways, to cleverly sidestep that and get you to results."

He argues that threat modeling is "more scary than it is hard." Ultimately, it's something every human being does all the time, from crossing the street to going shopping. If we break it down, there are only four simple questions we need to have in mind, he says, which are included in the threat modeling manifesto: "What are we working on?," "What can go wrong?," "What are we going to do about it?" and "Did we do a good enough job?"

Shostack wants everyone to contribute to threat modeling meetings, regardless of their experience. Therefore, the rules of these card games are specifically designed to force every player to jump in during each round, including those who come from a culture that discourages people from criticizing other people's ideas.

Hacking the rules of Cornucopia is sometimes allowed, however.

"[The game] is meant to be flexible, to be able to support anything," Irvine says. "We're giving people this permission to be playful. The idea is to drive that conversation. And there's nothing better than that one person on the team flaunting the rule."

Online threat modeling sessions like this one could get more popular soon. Next year, the process of examining potential security risks might become mandatory in many parts of the globe: The National Institute of Standards and Technology (NIST) recently issued the Recommended Minimum Standard for Vendor or Developer Verification of Code, which highlights the importance of threat modeling. It says that threat modeling should be done on multiple occasions during the development of a service or product, a suggestion that's likely to become the norm for every entity working with the US government, according to Shostack.

Still, the threat modeling game can be won both online and in real life, he says.

"The best way to win is actually to play because everyone wins," he adds. "The only losing move is not to play."

HOW THREAT MODELING CARDS BEGAN
The first physical version of Cornucopia was created by Colin Watson at the end of 2012. He was training Agile developers on how to identify security requirements for Web apps. Watson wanted to stay away from PowerPoint presentations and remembered the card game Elevation of Privilege (EOP), which Shostack created a few years before.

EoP follows the STRIDE (Spoofing identity, Tampering with data, Repudiation threats, Information disclosure, Denial of Service, and Elevation of privileges) methodology for threat modeling, but Watson wanted to adhere to the OWASP Secure Coding Practices. So he built a deck from scratch, tailoring it to the areas of interest to developers. Each card lists a type of attack and points players to additional resources.

"When I first created Cornucopia, I was concerned people might not know what some of these attacks were, but that's part of the learning experience," Watson wrote in an email. "I found it helped encourage participation by players if they had to ask and discuss what something meant during the analysis [game]."

Ideally, Cornucopia is played by three and six people, but one person can also play it alone. Still, there might not be enough perspectives if the group is too small. During threat modeling sessions, developers and security experts can use the entire deck or choose just a few suits, removing those that are not relevant to the issues they are discussing.