Data Mining the Radical Islamic Blogosphere
Blogs Study May Provide Credible Information
The Air Force Office of Scientific Research recently began funding
a new research area that includes a study of blogs.
By William J. Sharp / Air Force Office of Scientific Research Public Affairs
ARLINGTON, Va., June 29, 2006 – The Air Force Office of Scientific Research recently began funding a new research area that includes a study of blogs. Blog research may provide information analysts and warfighters with invaluable help in fighting the war on terrorism.
Dr. Brian E. Ulicny, senior scientist, and Dr. Mieczyslaw M. Kokar, president, Versatile Information Systems Inc., Framingham, Mass., will receive approximately $450,000 in funding for the 3-year project entitled “Automated Ontologically-Based Link Analysis of International Web Logs for the Timely Discovery of Relevant and Credible Information.”
“It can be challenging for information analysts to tell what’s important in blogs unless you analyze patterns,” Ulicny said.
Patterns include the content of the blogs as well as what hyperlinks are contained within the blog.
Within blogs, hyperlinks act like reference citations in research papers thereby allowing someone to discover the most important events bloggers are writing about in just the same way that one can discover the most important papers in a field by finding which ones are the most cited in research papers.
This type of analysis can help information analysts’ searches be as productive as possible.
The blog study is part of Air Force Office of Scientific Research’s new Information Forensics and Process Integration research program recently launched at Syracuse University, Syracuse, N.Y.
The new portfolio of projects consists of three areas of research emphasis – incomplete information and metrics; search, interactive design, and active querying; and cognitive processing.
One of the problems analysts may have with blog monitoring, Ulicny noted, is there is too much actionable information for the analyst to properly analyze.
“We are developing an automated tool to tell analysts what bloggers are most interested in at a point in time,” Ulicny said.
This analysis, Kokar said, is based on what Versatile Information Systems calls the RSTC approach to blog analysis – relevance, specificity, timeliness, and credibility. RSTC helps information analysts filter the most important information to study.
Of course, why exactly more research is needed five years after the definitive proof that it works - Able Danger and 9-11 - is beyond me, but still a welcome development.
Anyways, if you have not read "The One Percent Doctrine" by Ron Suskind, I strongly recommend it. There is nothing in the book about Able Danger, or military intelligence work, period, but here are some excerpts of interest about 2004. While terrorists learned years ago that phone calls and wire transfers were easy to track, they still rely on the web to talk and plan:
The front was shifting on the invisible battlefield. As financial and spoken signals intelligence shrank, the Internet became ever more the focus of the minions inside U.S. intelligence - just as it was for scores of young Islamic radicals, a "base" of men and women energized by images of U.S. troops mired in Iraq and Afghanistan and, since the spring, harrowing photographs of torture and humiliation at Abu Ghraib....
To draw a line between acquaintance dialogue and person-to-person communication, between suggestions and instructions, would be overwhelming if it were all in English and reviewed by cities full of freshly minted PhDs. Now, put it all in Arabic or any number of linguistic strains of the Near East of South Asia, and you have your arms around the magnitude of the challenge....
Some of the same sting operation techniques used in the Pacha Wazir storefront nearly two years before were now launched on the virtual frontier. Khan, it was soon discovered, was operating something of an Internet hub for al Qaeda out of Pakistan. In mid-June, he was placed inside a kind of digital terrarium of intense surveillance. In the basements of CIA and NSA, the grinders held their collective breath - as each e-mail, each IP number on each computer of each associate of Khan's began to light up the darkened battlefield.
If I had to guess, I would bet the CIA matrix is an outgrowth of the work Able Danger started and the chart with 55 pictures and descriptions - where Bush puts the big X's over the faces - is probably the chart that Preisser gave to Steve Hadley.
However, both of those are just guesses. Stay tuned. The 5th anniversary of 9/11 is less than two months away and should spark new interest in the Able Danger story.
Anyway, this quote from the preface of Suskind's book is pretty telling, too:
The vast federal government, under stress, does not work quite so efficiently as a single mind. It has protective urges, competing agendas, rules for who does what and who represents actions to the citizenry, the sovereign, the bosses; it accomplishes a great deal, yes, but is defined often by its dysfunctions. And that means it lies and dissembles, hides what it can, and sometimes acts out of self-preservation, because without your trust it is nothing but office space.
This has long been the case - a matter of life force trapped inside bureaucracies that everyone from Max Weber to Stephen R. Covey has fretted over - and maybe that's just something to be accepted, a point of resignation. Maybe, people don't really want to know about the internal disputes and roiling uncertainty, the dissonance. Or maybe the don't want to take on the turmoil and clarity that inhabit those on history's fault line - both the notables, who watch each active verb, and their fierce, frank invisible partners, whom you will now meet in the coming pages - as, side by side, they chase shadows on the cave wall of an enemy who is newly armed with destructive capability and sectarian certainty, and patience, and clever resolve, and maybe, tactical advantage.
In sleepier times, you could just go about your life and shrug, and say that there are mortgages to pay and children to school, sitcoms to watch, and that, from the start, two centuries ago, even some founding fathers felt the noisy rabble, beyond the ramparts, couldn't "handle the truth."
But these are not ordinary times. Knowledge, in fact, is power, enough to burn off fear. And you at least ought to know what the hell's been going on.
It's what Americans do.