Security Lessons From Protecting Live Events

Live events such as concerts and sports games are generally chock-full of action, both on the field and behind the scenes. IT and security teams managing these venues navigate a complex environment that includes a traditional corporate infrastructure, special equipment required for the event, a large army of suppliers and contractors, and all of the devices brought in by spectators. And the prospect of a cyberattack during the actual event always looms large.

Securing live events can be the most complicated and most complex to protect, says Karim Benslimane, director of cyber intelligence at Darktrace. Running a stadium can be split into a “business perspective,” which is akin to managing the technology infrastructure for a very large enterprise, and the “event perspective,” which includes all the people and equipment involved with setting up and hosting the event itself.

“When there is no event doesn’t mean the IT guys don’t have anything to do,” Benslimane says. “They are already busy running the business.” He speaks from experience. Prior to joining Darktrace, he was head of information and communications technology and cybersecurity for various international venues and events.

Understanding the Cyber Paradox

The business infrastructure depends on much of the same technologies and enterprise applications – Active Directory, telephony, wired and wireless networking, ERP, CRM, databases, and e-commerce applications, to name just a few – as a large organization  requires to manage the day-to-day operations. Payroll needs to make sure people are paid, employees need to be able to communicate internally and with clients and customers, and tickets have to be printed and sold.

The event has its own infrastructure, and managing the event requires a completely different mindset because of what Benslimane refers to as the “cyber paradox.” The traditional security mindset is to deploy technology and control to add layers to make it harder for attackers to access systems and restrict what is added to the network. Setting up an event, by definition, involves hiring suppliers and contractors (along with their sub-suppliers and subcontractors) who bring their own assets and need access to the venue’s infrastructure, such as VoIP and Wi-Fi. For example, the venue may own the jumbo screens, but the immersive elements and special effects that make the event come alive are pushed onto the screens from third-party systems.

“From a cybersecurity point of view, it’s obviously a paradox where you are trying to secure your assets on the left, but on the right, you are forced to allow external people and external devices to come physically into your venue and use your infrastructure,” Benslimane says. It doesn’t make sense to talk about securing the perimeter when people are already inside, he notes.

On top of that is the live event itself, or “D-Day,” which requires letting even more people physically enter the venue with their own devices and access services such as Wi-Fi.

“It was a big challenge. You have nothing to control and protect you,” Benslimane says.

Limited Time to Respond

As the saying goes, the show must go on. Security teams in the events industry have to “rush and run” to mitigate security incidents as they occur so that the event can go off as scheduled and associated services (such as broadcasts and streaming contracts) are not impacted.

“If something happened during the 100-meter men’s final [at the Olympics] or the [FIFA] World Cup final, think about the brand impact,” Benslimane says. Interrupting the broadcast would mean millions of TV and screens will see a black screen, which would result in immense financial losses for the brand. There will be penalties from lost advertising. This would also affect the host country’s reputation.

Those were the same challenges Wired reporter Andy Greenberg wrote about in his book about Olympic Destroyer, the malware that knocked out the domain resolvers during the opening ceremonies. As a result of the attack, some attendees couldn’t print tickets to enter the stadium, the Wi-Fi network was offline, TVs around the stadium and other facilities could not show the ceremony, the RFID-based security gates for Olympic facilities were not working, and the official Olympic app was broken. If the systems remained offline after the opening ceremony ended, athletes, visiting dignitaries, and spectators would find they had no access to the Olympics app full of schedules, hotel information, and maps. 

“The result would be a humiliating confusion,” Andy Greenberg wrote for Wired.

Attackers realize that events can’t be delayed and take advantage of that time constraint to cause the most damage, Benslimane says. There is no time to recover when the match ends in a few minutes.

Holding the Event Hostage

Ransomware attacks encrypt documents and make it hard for organizations to function. A ransomware attack on a live event cripples the technology and interrupts the show. Cybercriminals clearly understand that the venue cannot postpone, or replay, D-Day. Imagine if a cyberattack disrupted the systems to the point that the game cannot continue, and the venue asks the crowd to leave and come back next week. 

“Just imagine the fights,” Benslimane says.

Organizations rely on the metrics MTTK (mean time to know) and MTTR (mean time to repair) to identify, investigate, and mitigate the incidents. Those metrics highlight the challenges facing live events because the clock is already ticking. If the security team takes more than 10 or 15 minutes to detect and investigate an incident, that is already halfway through the first half of the game and the halftime show is coming up, Benslimane says. This is the biggest advertising window, and not being able to broadcast the ads would be a significant blow to the organization, which makes it even more likely that these venues would pay the ransom.

“The cybercriminals clearly understand that if something happens during D-Day, there won’t be enough time to respond,” Benslimane says.

Moving Faster than the Attackers

Attackers move faster than defenders, which makes it difficult for security teams to respond during an event. For defenders to be able to respond effectively, they need to be operating at machine speed, which means using artificial intelligence (AI) as part of the security response, Benslimane says, noting that he deployed Darktrace’s technology in his past roles. The AI can look at all of the events happening in the network and correlate them quickly to identify which issues are problematic. If the investigation finds that the issue is actually not a problem, then the AI doesn’t take any action. And if there is an issue, the AI already has the information it needs from the investigation to neutralize the threat and remediate, such as by blocking a particular connection or isolating the affected system.

The AI is like a “sponge” for knowledge. 

“As long as you put in water, the sponge will absorb it. The AI brain will absorb everything you give it,” Benslimane says.

Speed is paramount for detection, investigation, and making the decision on how to respond, especially when the environment is complex and constantly evolving. The infrastructure for a live events venue is not static, so the AI is constantly learning, analyzing, and making decisions.

The AI is “much faster than 1000 cybersecurity guys in a SOC – even if it’s the best SOC in the world,” Benslimane says.