« NSA Confirmed It Performed Warrantless Searches Of U.S. Citizens Phone Calls And Emails | Main | Surprise! Metadata About Your Online Activity Reveals Where You've Been »

Monday, April 07, 2014


Feed You can follow this conversation by subscribing to the comment feed for this post.

Chanson de Roland

While no one knows why the average of a large group of people, who use only open-source intelligence, is better at predicting world events, statistics provides an answer, which is not complete satisfying but which is probably true: The wisdom of a sufficiently large crowd, that is, a crowd of adequate sample size for the level of confidence, corrects for error by random sampling, so that what is left is the average, which is the best true estimate of human judgment.

In addition, as Mr. Jenkins writes, supra, the humans in the sample don't try to collect all information, much less analyze all information, but use various strategies for focusing on what they judge to be relevant information. Some of those collective strategies will be great, others good, and some awful, but random sampling will find the best estimate of all those strategies, with the errors canceling each other out, as Professor Tetlock explains in the article.

So the problem for the CIA et al. is that even if they opt to use various strategies to identifying and analyzing relevant information, it will be limited by its number of analysts. What CIA may do is use the best strategies from GJP and ACE to enhance their predictive success.

However, whether any of those strategies is amenable to being reduced to an algorithm or whether the CIA must train its analysts to use them will be interesting. But using keywords and pattern recognition techniques to scan everything is of dubious value, especially since our adversaries know that we are doing it. So it is well past time to enhance our predictions by using the best strategies for identifying and analyzing relevant information and/or the collective judgments from sufficiently large samples of people, who use open-source intelligence and their various strategies and intuitions to make predictions.

The comments to this entry are closed.


  • Updates via E-mail RSS Feed Updates via Twitter Updates via Facebook


  • Bloggers' Rights at EFF
  • George Jenkins, author of the I've Been Mugged Blog


  • © 2007 - 2017. George Jenkins. All Rights Reserved.


  • <$MTStatsScript$>