June 12, 2012
Detecting Disasters: Social-Media Scanning Tool Could Lead to Quicker Crisis Response
During natural disasters, emergency officials have increasingly tapped social networks like Twitter and Facebook to deliver information, recruit and deploy volunteers, and manage response and recovery efforts. Analysts in APL's Research and Exploratory Development Department (REDD) have created a tool that may help first responders get an even faster jump on a developing crisis.
The technology automatically detects topics of discussion that seem out of place or anomalous, on social media sites, explains REDD's Jaime Montemayor. "We have observed that these anomalous topics—we call them 'social signals'—often coincide with important events," he says. "For example, our algorithms know very quickly that a natural disaster like a tornado, or a man-made crisis like a bombing, is occurring, simply by identifying unusual patterns in the social signal."
Montemayor, with REDD's Evan Sultanik and Clayton Fink, recently examined thousands of tweets surrounding three incidents that occurred in 2011: the February 22 earthquake in Christchurch, New Zealand; the April 27 tornado in Tuscaloosa, Alabama; and the July 13 bombings in Mumbai.
Fink developed the data-collection algorithms that solved a number of issues, including the messiness of Twitter data, the ambiguity of locations, and the multistage collections to enhance the quality and volume of the data. What they discovered is that an interruption from the norm shows up in posted comments.
Expressions on Twitter tend to follow "natural" rhythms, Sultanik explains. "The rhythm can reflect the day–night cycles, or the exuberant utterances before, during, and after sporting events. When an unexpected event occurs in the midst of these natural rhythms in the physical world, some expressions [terms] will appear and flow counter to everything else."
This is data mining with a unique twist, Montemayor says. "The traditional approach to solving these types of detection and data-mining problems is to analyze the semantic content of the text—that is, having the computer try and figure out what the humans are saying," he explains. "Our approach is novel in that we are not looking at the meanings of words; instead, we are just looking at the frequencies of their usage. It turns out that, with enough data, the frequencies of usage alone are enough to detect which words are being used abnormally, and in what combinations. The fact that we do not use meanings has the added benefit of allowing this method to work with any language."
The team happened upon the project while working on an independent research and development effort on the spread of rumors in social media. "We know from the literature that one way people make sense of events is through the generation and spread of rumors, and we wanted to develop algorithms and visual analytic workflows that could capture and identify rumors," Montemayor says. "By accident, we discovered another 'signal' that was occurring at the same time as natural disasters. We quickly saw applicability to emergency management applications."
Beyond security and relief efforts, the software has potential commercial applications as well. "Our algorithms can also be used to track brands and identities," Montemayor says. "For example, our tool could detect emergent, negative rumors about a targeted commercial product."