A paper by Jeffrey R. Cooper on “Curing Analytical Pathologies” (pdf) that was withheld from the CIA web site but posted on the Federation of American Scientists web site last week has now been downloaded tens of thousands of times, suggesting that there is widespread interest in a critical assessment of intelligence analysis.
One of the analytical techniques cited favorably by Cooper (at pp. 48-49) is called “Analysis of Competing Hypotheses” (ACH).
More information about this structured, methodologically rigorous approach to intelligence analysis was presented in a January 2000 paper (pdf) by Air Force MSgt Robert D. Folker, Jr. that was published by the Joint Military Intelligence College. The author compared it with less formal approaches and found that it offered significant advantages.
“At the heart of this controversy is the question of whether intelligence analysis should be accepted as an art (depending largely on subjective, intuitive judgment) or a science (depending largely on structured, systematic analytic methods).”
“Resolving this question is necessary to provide direction and determine an efficient and effective approach to improve analysis,” wrote MSgt. Folker.
“If qualitative intelligence analysis is an art, then efforts to improve it should focus on measuring the accuracy of one’s intuition, selecting those analysts with the best track record, and educating them to become experts in a given field.”
“If, on the other hand, qualitative intelligence analysis is a science, then analysts should be trained to select the appropriate method for a given problem from a variety of scientific methodologies and exploit it to guide them through the analytical process,” he wrote.
Based on empirical tests, the author found reasons to conclude that there is indeed a “scientific” dimension to intelligence analysis that has been neglected, and that intelligence analysis would benefit from more structured approaches.
See “Intelligence Analysis in Theater Joint Intelligence Centers: An Experiment in Applying Structured Methods” by MSgt Robert D. Folker, Jr. (USAF), Joint Military Intelligence College, January 2000.
This entry was posted
on Monday, May 8th, 2006 at 1:21 pm and is filed under Secrecy.
You can follow any responses to this entry through the RSS 2.0 feed.
You can skip to the end and leave a response. Pinging is currently not allowed.
PDF 
May 8th, 2006 at 3:40 pm
Even with science acting as the backbone of our intelligence analysis, there must be a consistant judgement that not only proves to be strategic but to also serve as judgement that is willing to adapt to obstacles that are forever changing. Without superb judgement, our sciences would not evolve into the more advanced methods of problem solving that we have today. Advanced judgement not only serves to advance our sciences but also serves to enforce these methods or to question them. The human mind should always be active in ways other than to solve a scientific hypothesis or mathematical equations. I do not believe that good judgement will ever be substituted by any kind of what we would consider as an advanced method of problem solving. Both science and intellectual intelligence must work together in order to create answers to the difficult questions that face our intelligence community.
May 17th, 2006 at 9:18 pm
There is no disagreement between Brendan’s comments and my paper, which was graciously made available by FAS. In fact I make the same argument — that good intelligence analysis is a mix of both art & science. I provide evidence in my paper that certain analytic tools can help the analyst put specific intelligence problems in the proper analytic framework, not by eliminating the role that intuition can play in analysis, but by exposing subconscious mental shortcuts and ensuring that the problem is systematically, thoroughly, and sufficiently analyzed to include all possible alternatives. Without using these structuring techniques, analysts will often jump to conclusions and only look for data that supports their initial belief ignoring contradictory evidence. These tools provide a means to structure a problem; they are not a substitute for thinking but can enhance discussion and thought. Ultimately, one must be accountable for one’s judgments — the quality of the analysis depends on the soundness of one’s thinking.