ISPs Team In Bot Cleanup

It used to be that Internet service providers (ISPs) kept either a low profile or a hands-off approach to bot infections in their networks. But ISPs have gradually been getting more aggressive in the cleanup effort -- including a new best practices initiative by an ISP industry group.

The Messaging Anti-Abuse Working Group (MAAWG) recently published guidelines and best practices (PDF) for mitigating large-scale bot infections in residential networks, representing the group's first major foray into the bot battle. ISPs traditionally have been criticized for distancing themselves from the bot problem, but ISPs say times are changing.

"It used to be, we provide you a pipe and get the heck out of the way. But more and more, ISPs seem to be realizing they have a vested interest in making sure customers are protected in offering them services and online protections to do that," says Chris Roosenraad, co-vice chairman for MAAWG and a principal engineer for a major U.S. ISP. "ISPs interviewed a couple of years ago would've said, '[Bots] are a customer's problem, not ours.' Nowadays, they [know] they have the responsibility to protect their network, if nothing else."

ISPs have always had the challenge of balancing their users' privacy and network access with protecting the security of the network, which hasn't made it easy for them to take a bigger role. But bandwidth and brand reputation concerns -- along with the reality of botnet proliferation -- have forced ISPs into the spotlight.

"They've always been working on the problem, but didn't want to be high profile because it made it difficult for them to remediate the issues," says Adam O'Donnell, director of emerging technologies at Cloudmark.

O'Donnell says it has been tough for ISPs to find solutions for cleanup that scale to their networks. "A lot of products claiming to be botnet mitigation strategies either don't work or only work up to SMB scale," he says. "ISP scale has to be far more efficient."

Many ISPs offer free antivirus scanning and firewalling to their residential users, but, meanwhile, the bot count keeps growing: Bots account for about 90 percent of all spam, and bot malware now hits mobile devices and non-Windows platforms. Today's Twitter attack, which knocked the service offline temporarily and continued to cause service disruptions, is also a chilling reminder of how powerful botnets can be in waging distributed denial-of-services (DDoS).

But just how deep ISPs can drill down into their customers' bot-infected machines depends on where they are in the world, as well as their terms of service. "In Europe, customer privacy is a very high priority and is legally enshrined" in some aspects, Roosenraad says. That makes it tricky for ISPs to take a more proactive approach to getting their customers to clean their machines.

And even with their more public cleanup efforts, ISPs are still basically reacting to bot infections, not proactively halting them in their tracks. "There's not a lot we can do to prevent the infections," Roosenraad says. "That would get into customer privacy and could be incredibly invasive about what's done on [their] networks," he says.

Meanwhile, MAAWG's new guidelines for ISPs comprise three phases: detection, notification, and remediation. The best practices are a conglomeration of ISP members' methods and suggestions.

"This is the first time ISPs have stood up and said, 'This is what we think we should be doing about the problem,'" says Michael O'Reirdan, chairman of MAAWG and a distinguished engineer in national engineering and technical operations at a major U.S. ISP. "What you're seeing now is the first real, significant public focus on what ISPs can do or are doing here."

Identifying a bot in an ISP network is trickier politically than technically: The ISP has to maintain its users' privacy while not blocking or slowing legitimate traffic and not tipping off the botnet operators that they have found infected machines under their control, according to the MAAWG guidelines. They recommend a combination of network traffic inspection tools and input and feeds from other ISPs and customers. And DNS lookups can be tracked on the infected machine to look for blacklisted domains and IP addresses.

In some countries, ISPs can scan a customer's IP space for unpatched or vulnerable computers, the guidelines say.

ISPs should notify their customers with bot infections directly or using a "walled garden" or in-browser notification. A walled garden is a restricted area on the network where the infected machine can be placed while the user is alerted and his machine cleaned up. "They aren't allowed to surf beyond where the ISP wants them to," O'Reirdan says.

ISPs can also display an infection alert message in an iFrame or in part of the Web browser content to let the user know he or she is now a bot. The MAAWG guidelines say while this is an ideal way to alert a user whose machine is a bot, it's also "the most costly option."

Remediation, meanwhile, should be built around a "well-publicized" security portal where the bot-infected user can be sent to clean his machine. The portal would have educational information on what a bot is, as well as links to tools for cleanup.

ISPs say the education piece of the puzzle is crucial, especially since 80 percent of consumers are aware of bots, but only 20 percent believe they will get infected, according to a survey MAAWG released last month.

Cloudmark's O'Donnell says the next step for ISPs will be some form of HTTP filtering to prevent Web-borne bot infections. "But that's a technology that's not ready to scale for them yet," he says.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.