San Francisco Team Solves DARPA Shredder Challenge

Obama's Tech Tools

Custom-code computer-vision algorithms helped a San Francisco-based team solve a challenge by the Defense Advanced Research Projects Agency (DARPA) to solve complex puzzles comprised of shredded documents.

Thirty-three days after DARPA unveiled the so-called Shredder Challenge, a team of three programmers called "All Your Shreds Are Belong to U.S." pieced together the five documents in the challenge, beating out nearly 9,000 other teams to claim the $50,000 prize.

The team spent nearly 600 hours creating algorithms to assemble the documents, which were shredded into more than 10,000 pieces. The team developed algorithms to suggest fragment pairings and then manually verified the pairings to piece together the documents--which in the end turned out to have a common theme (PDF).

[DARPA is trying a number of unusual tactics. Read DARPA Investigates Storytelling As Security Science..]

The theme running throughout the puzzles was Antonio Prohias, creator of a comic strip that debuted in Mad Magazine in 1961 called "Spy vs. Spy." The strip spawned a television show and other commercial paraphernalia.

Given the difficult nature of the challenges, DARPA organizers said they were surprised not only that all of the puzzles were solved, but at the relatively short time it took to solve them.

"Lots of experts were skeptical that a solution could be produced at all let alone within the short time frame," said Dan Kaufman, director, DARPA Information Innovation Office, in a press statement.

He said that the most effective approaches to solving the puzzles in the end were a combination of computational tools, crowd-sourcing, and "clever detective work."

"We are impressed by the ingenuity this type of competition elicits," Kaufman said.

DARPA introduced the challenge Oct . 27, with the goal of reassembling five shredded documents of increasing difficulty. The first two problems contained 224 and 373 pieces, respectively; puzzle three had 1,115 pieces; puzzle four had 2,340 pieces; and puzzle five had 6,068 pieces.

The practical aim of the challenge was to develop ways to reconstruct shredded documents that U.S. soldiers come across to use to gather intelligence information. The Department of Defense (DOD) also wants to try to figure out how enemies might be reassembling U.S. documents by crowdsourcing document unscrambling.

The Obama administration has turned to crowdsourcing and challenges to help the feds solve technological and other problems. The administration has even launched a website called Challenge.gov on which agencies can post challenges for people to solve for a range of cash prizes.

Our annual Federal Government IT Priorities Survey shows how agencies are managing the many mandates competing for their limited resources. Also in the new issue of InformationWeek Government: NASA veterans launch cloud startups, and U.S. Marshals Service completes tech revamp. Download the issue now. (Free registration required.)