Representative Mac Thornberry’s proposed Acquisition Agility Act addresses several aspects of the ongoing military acquisition debate, including prototyping. Seen as a possible tool to test nascent technology before production and reduce concurrence issues of having to arbitrage technology while building a weapons system, prototyping has become increasingly popular for fighting delays and cost growth. There is evidence that prototyping can be used to improve acquisition outcomes and reduce cost growth and schedule slippage. But there is also evidence that improper implementation can actually exacerbate these issues, and that the use of prototypes must rely on knowledge of what problems prototyping fixes, what it does not, and aligned incentives.
The statistical record of prototyping military weapons systems is mixed. Studies about whether such efforts actually reduce cost growth have shown ambiguous results. However, the RAND Corporation found that large and less mature programs did seem to benefit from prototypes. Digging deeper into the results, RAND concluded that prototyping can be beneficial overall, if conditions merit prototyping. The conditions center on the technology being used, the size and scope of the project, and the ability of decision makers to properly implement the lessons learned from the prototyping stage.
While testers may want to plug into the prototype everything they want to test, prototypes are useful for testing several critical technologies, not every aspect of a planned program. Too many technologies or systems being tested complicates the results and could balloon the costs of the prototype. Some technologies may not necessarily need to be prototyped, and not all systems are best tested in such a manner. Instead, testers should focus on resolving the most vital technological uncertainties. These tests should also be undertaken in as realistic environments as possible.
If prototyping is used, then the program should make sure to use the results in the final product. This may seem relatively straightforward, but that has not always been the case. Requirements creep can alter the final product and reduce the value of the information gleaned from the prototype. Stakeholders in the process should also be willing to cancel programs if prototypes reveal that the specific approach will not work or that the technology is immature. There should be no commitment to production during prototyping.
Prototyping can yield significant benefits, from validating useful technology to increasing stakeholder confidence in a particular platform. The generation of information can be used to reduce the overall costs of the platform. But, when a prototype demonstrates that a particular idea will not work as hoped, then stakeholders need to be willing to step back and not consider sunk costs. If they don’t, the costs of building an expensive prototype will be followed by increased costs to redevelop the platform, rather than the savings that would come from canceling it.