GenAI-Powered Attacks Change the Game

If there was ever a technology buzzword that drove more intrigue, polarization, and old-fashioned coattail riders than the phrase "generative AI," I haven't heard it yet. It's everywhere. But more importantly for defenders, generative artificial intelligence (GenAI) is now a friend of the enterprise, which of course also means it's quickly becoming a sidekick for attackers looking to advance their course across hybrid environments. So, for defenders, does that mean the only way to fight AI is with AI?

To better understand how GenAI is being used by attackers and what that actually means, I'll look at it from a few different angles. First, I'll quickly level set on how GenAI is being used across enterprises and why that can give attackers an advantage, then I'll dig into what GenAI usage means for attacks, and finally I'll lay out why all this matters for defenders, so they can adapt.

Enterprise Adoption of Generative AI

The most common usage of GenAI across organizations today is in the form of large language model (LLM) chatbots such as Google Gemini, Meta.ai, ChatGPT from OpenAI, and Microsoft Copilot. Copilot is the most common because it's integrated into Microsoft 365 and can be used alongside its family of productivity apps. In fact, according to our data, around 40% of the enterprises Vectra AI monitors use Microsoft Copilot. This isn't too surprising considering the widespread enterprise adoption of Microsoft 365 and the practicality of Copilot to help increase the productivity, efficiency, and speed of the apps that workers use each day.

For example, employees can use Copilot to prompt quick searches for documents, images, and specific terms across Microsoft 365 apps. It can be used in Microsoft Teams to summarize meetings and in Microsoft Word to create drafts, ideas, and revisions. There's no denying the practicality of it, and it's going to get more integrated, more useful, and more powerful — which unfortunately means it will become more powerful to attackers as well.

Everyone Is Getting More Productive With Gen AI, Including Attackers

As defenders, we know attackers will use anything they can to advance attacks, making it clear that we must address this new attack surface or vector. We've seen attackers use LLMs to aid in phishing and figure out what to do for reconnaissance, as well as enhance the ability to do scripting or social engineering — this is really just the tip of the iceberg of what attackers are starting to do. In fact, MITRE currently has 56 documented techniques in its ATLAS Matrix showing the machine learning (ML) technique that belongs to each.

This indicates that GenAI is truly becoming a force multiplier for everyone. Just as enterprise employees are using tools like Copilot or Gemini to remove latency from their day-to-day work, attackers are doing the same with the GenAI available to them. But in both cases, GenAI doesn't replace humans — it just makes them more efficient.

For example, an attacker using GenAI to help advance an identity attack must still take all the same steps. They will still have to register a device, sign in, and move laterally, even when using AI — and all these actions are still detectable. The important thing for defenders is the speed we can detect post-compromise behaviors, as the attacks using AI are moving much faster.

For example, with an identity attack, GenAI can help craft phishing emails that are more convincing, making it easier to bypass multifactor authentication (MFA) and sign in with an identity. Once inside, they may leverage available internal GenAI tools to start asking questions like "how to register a device," which, if successful, could provide persistence to advance further. They could even gain passwords by using the right phrasing. What was once a sophisticated attack has now become simple for novice-level attackers. Not just from a speed standpoint, but from an overall barrier of entry.

What Does GenAI Usage Mean for Defenders?

There's no mistaking that GenAI provides value to the enterprise by way of productivity. We are also at the point where attackers are bringing it into their fold as well — as they do with any new tech with a high amount of usage. For defenders, this means we have to be fast. Attacks on hybrid enterprises are now turbocharged, and we need to be able to stop them no slower than in real time.

The good news is AI cybersecurity tools are also making it faster and easier to investigate and respond to incidents. With the right approach, security operations center (SOC) teams can prioritize LLM activity (or any threat across any surface) and know exactly what identities are doing without heavy investigation. We can use it to understand who is using Copilot (or other tools) even before a user gets compromised. We can learn who the power users are, what type of data they're interacting with — all the things that help a defender prepare to stop an attack. We understand where potential exposure exists, where an attacker is abusing GenAI. Are they using Copilot to gain access to Entra ID or AWS? We can find out, while automatically prioritizing the threat.

When you think about what makes a defender's job challenging today, it's continuously increasing exposure (GenAI is a great example), latency that comes from too many disparate tools, and noise created by false positives. When AI is used right, it can greatly reduce all those obstacles — making it possible for defenders to prioritize the most critical risks, automatically lock down infected hosts and accounts (identities), and more importantly, have the time they need to stop an attack in progress. We're early in the GenAI journey, and attackers weren't first to the party — let's keep it that way.

By Mark Wojtasiak, Vice President of Research and Strategy, Vectra

About the Author

Mark Wojtasiak is vice president of research and strategy at Vectra. Passionate about security research and strategy, he has 27 years of experience in IT, making him a security product marketing leader in the industry.