This is excellent research, from Stephen Biddle, Jeffrey Friedman, and Jacob Shapiro, at International Security, "Testing the Surge: Why Did Violence Decline in Iraq in 2007?"
At issue: What explains the decline of military and civilian fatalities in Iraq after mid-2007? The decline in violence coincides with the Bush administration's high-profile shift in war strategy, popularly called "the surge." Opponents of the war dismissed the administration's claims that the decline in violence was the result of a successful military reorientation under General David Petraeus, who combined increased troop contingents with a new war-fighting doctrine that sent patrols out into the most dangerous Baghdad neighborhoods to clear and hold the areas most wracked by sectarian violence. Troops were dismounted and mobile and military bases were dispersed, in contrast to pre-surge war-fighting that stressed large, fortified bases and mounted troop patrols. The antiwar opponents argued instead that sectarian violence was so unchecked that there remained no more ethnic groups left to cleanse. Ethno-religious rivalry played out between Sunni and Shiite Muslim factions. According to the authors:
Proponents of the cleansing thesis argue that it was the spatial intermingling of prewar Sunnis and Shiites that led to violence: large, internally homogeneous communities would be defensible and thus secure, but the prewar patchwork quilt of interpenetrated neighborhoods created a security dilemma in which each group was exposed to violence from the other. In this view, the war was chiefly a response to mutual threat, with each side fighting to evict rivals from areas that could then be made homogeneous and secure. While the populations were intermingled, the violence was intense, but the fighting progressively unmixed the two groups, yielding large, contiguous areas of uniform makeup with defensible borders between them. This in turn resolved the security dilemma, and as neighborhoods were cleansed, the fighting petered out as a product of its own dynamics rather than as a response to U.S. reinforcements [p. 14].Political bloggers will remember these debates quite well, which makes this research especially interesting. It provides a careful empirical rebuttal to the debased arguments of the antiwar left, groups who worked to politically destroy the Bush administration, and often gave aid and comfort to the enemies of the United States.
The authors demonstrate that the cleansing thesis, despite its intuitive appeal, cannot explain the reduction in violence over the time period. However, it wasn't just the surge alone that prepared the way for the military victory in Iraq. The authors indicate that a complex interaction took place between the surge of military force (and the change in troop deployments) and the rise of what's been called the "Anbar Awakening" --- the mobilization of local Sunni tribal forces in an uprising against the insurgency of al Qaeda in Iraq. Antiwar opponents also latched onto the Awakening thesis as a means to deny the Bush administration credit for the improvement of conditions on the ground. If local tribesman rose up against outside forces, aided by cash payments (amid the decline of sectarian violence, since everything was all cleansed out), then it wasn't more troops or the innovations of the COIN doctrine. It was local contingencies, and the Bush adminstration was not only wrong about the war, but its top officials should be tried as war criminals.
But the authors show that there was synergy between the surge and the Awakening, and that military improvement would not have taken place without the synergistic interaction of these two variables. Folks will want to read the piece for the full argument and evidence. The authors employ historical process-tracing analysis combined with a statistical data set charting the "standing up" of the Sons of Iraq forces (SOI). From the article:
The surge-Awakening synergy thesis ... sees the reinforcements and doctrinal changes as necessary but insuficient. In this view, the surge was too small, and the impact of doctrinal changes insufªcient, to defeat a determined insurgency before the reinforcements’ time limit was reached and their withdrawal began. Hence the surge without the Awakening would have improved security temporarily but would not have broken the insurgency, which would have survived and returned as the reinforcements went home. The surge added a temporary, yearlong boost of about 30,000 U.S. troops to a pre-surge coalition strength of about 155,000 foreign and 323,000 Iraqi troops and police as of December 2006 (Iraqi Security Forces, or ISF, grew by about another 37,000 by September 2007, when violence had begun to drop). Thus the surge entailed only a marginal increase in troop density: an expansion of less than 15 percent overall and perhaps 20 percent in U.S. strength. Half of the overall increase, moreover, was in Iraqi forces, which were far from proªcient in the new U.S. methods by 2006–07.The full article is here.
And as mentioned above, the U.S. component had only about a year in which to function at this strength, after which it was to return to pre-surge numbers or fewer. For this reinforcement per se to have been decisive, one must assume that previous troop density lay just below some critical threshold that happened to be within 20 percent of the presurge value. Although this coincidence cannot be excluded, there is no prima facie reason to expect it.
For synergy proponents, the Awakening was thus necessary for the surge to succeed. In this view, the Awakening had three central effects. First, it took most of the Sunni insurgency off the battleªeld as an opponent, radically weakening the enemy. Second, it provided crucial information on remaining holdouts, and especially AQI, which greatly increased coalition combat effectiveness. And third, these effects among Sunnis reshaped Shiite incentives, leading their primary militias to stand down in turn.
As for the first two points, although the SOI movement never comprised just former insurgents, the insurgency nevertheless provided much of the SOIs’ combatant strength—and the bulk of the secular Sunni insurgency nationwide became SOIs over the course of 2007. By the end of the year, SOI strength nationwide had reached 100,000 members, under more than 200 separate contracts. As insurgents progressively realigned in this way, the remaining insurgency shrank dramatically. The fact that so many SOIs were former insurgents also made the SOIs uniquely valuable coalition allies: they knew their erstwhile associates’ identities, methods, and whereabouts in ways that government counterinsurgents rarely do. When insurgents who had been allied with AQI realigned as Sons of Iraq, the coalition suddenly gained intelligence on AQI membership, cell structure, the identity of safe houses and bombmaking workshops, and locations of roadside bombs and booby traps. Guerrillas rely on stealth and secrecy to survive against heavily armed government soldiers. When SOIs lifted this veil of secrecy, coalition ªrepower guided by SOI intelligence became extremely lethal, creating ever-increasing incentives for holdouts to seek similar deals for themselves; soon only committed AQI fanatics remained, marginalized in a few districts in Iraq’s northwest.
In the synergy account, Sunni realignment in turn had major consequences for Shiite militias such as the Jaish al-Mahdi. Many of these militias began as self-defense mechanisms to protect Shiite civilians from Sunni attack, but they grew increasingly predatory as they realized they could exploit a dependent population. Rising criminality in turn created fissiparous tendencies as factions with their own income grew increasingly independent of their leadership. When the SOIs began appearing, the Sunni threat waned, and with it the need for defenders. At the same time, the SOI cease-fires freed arriving U.S. surge brigades to focus on Shiite militiamen. These developments created multiple perils for militia leadership. In previous firefights with U.S. forces, the JAM in particular had sustained heavy losses but easily made them up with new recruits given its popularity. Shiites’ growing disaffection with militia predation, however, coupled with declining fear of Sunni attack, threatened leaders’ ability to make up losses with new recruits. At the same time, intraShiite violence among rival militias, especially between the Badr Brigade and the JAM, posed a rising threat from a different direction. When Shiites were united by a mortal Sunni threat and U.S. forces were tied down by insurgents and AQI, these internal problems were manageable. But as the Sunni threat waned, Shiite support weakened, internal divisions multiplied, and U.S. troop strength grew, Shiite militias’ ability to survive new battles with coalition forces fell. In the synergy account, these challenges persuaded Muqtada al-Sadr to stand down rather than risk another beating from the coalition, and the result was his announced cease-ªre of August 2007—which took the primary Shiite militia off the battlefield, leaving all of 2006’s major militant groups under cease-ªres, save a marginalized remnant of AQI, and producing the radical violence reduction of late 2007 and thereafter.
Proponents of the synergy thesis thus see the Awakening as necessary for the surge to succeed. In this view, however, neither the surge nor the Awakening was sufficient, nor did these factors combine in an additive way. As noted above, Sunni groups had attempted similar realignments on previous occasions—and those earlier attempts had all failed at great cost. For the synergy school, what distinguished the failures from the successful 2007 Awakening was a coalition force that could protect insurgent defectors from counterattack. The surge may not have been large enough to suffocate a determined insurgency, but it was large enough to enable cooperation with turncoat Sunnis and exploit their knowledge to direct coalition firepower against the still-active insurgents, enabling them to survive the kind of retaliation that had crippled their predecessors... [pp. 23-26]
The authors caution against applying the lessons from their research to the war in Afghanistan. The correlation of factors in Iraq were highly idiosyncratic, and not likely to be replicated elsewhere. But the authors do indicate that much remains to be teased out on theories of counterinsurgency, that much more work along these lines awaits, which in turn will provide important information for policymakers.