Intelligence Agencies Seek Tools To Predict Global Events

Inside DHS' Classified Cyber-Coordination Headquarters

The research arm for federal intelligence agencies is seeking to mine data from websites, blogs, social media, and other public information to help it better forecast global events.

The Intelligence Advanced Research Project Agency's (IARPA) Open Source Indicators (OSI) Program aims to develop technology for "continuous, automated analysis" of data from myriad Web-based sources to predict when events such as disease outbreaks, political and humanitarian crises, riots, economic instability, mass violence and resource shortages may occur, according to a solicitation on its website.

"Many significant societal events are preceded and/or followed by population-level changes in communication, consumption, and movement," according to the solicitation. "Some of these changes may be indirectly observable from publicly available data, such as Web search queries, blogs, micro-blogs, Internet traffic, financial markets, traffic webcams, Wikipedia edits, and many others."

IARPA's interest in the program is based on research that claims these data sources are useful for early detection of events, but that currently there aren't methods for aggregating and integrating data available from multiple sources to come up with accurate predictions.

Those interested in responding to IARPA's request for information should be prepared to offer information for developing methods for using population behavior change in anticipation of and response to events of interest, and for processing publicly available data that reflect those population behavior changes.

IARPA also is looking for ideas to develop data-extraction techniques focusing on volume rather than depth, and an innovative use of statistical methods to fuse combinations of time series to generation event warnings, among others.

IARPA's interest in developing better event-prediction technologies is not new. The agency already is developing software that can aggregate knowledge from different analysts and generate probable forecasts for a range of events through the Aggregative Contingent Estimation (ACE) program.

Using social media data for analysis purposes also is an interest of the Defense Advanced Research Project Agency (DARPA), the Department of Defense's (DOD's) research arm.

Through a program called Social Media in Strategic Communication, DARPA wants to use social media activity to help the DOD better understand the environment in which it operates and use information more nimbly to support its missions.

Attend Enterprise 2.0 Santa Clara, Nov. 14-17, 2011, and learn how to drive business value with collaboration, with an emphasis on how real customers are using social software to enable more productive workforces and to be more responsive and engaged with customers and business partners. Register today and save 30% off conference passes, or get a free expo pass with priority code CPHCES02. Find out more and register.