How do we create military resilience?

Today's quote is from page 76 of Donella Meadow's Thinking In Systems:

 "Resilience arises from a rich structure of many feedback loops that can work in different ways to restore a system even after a large perturbation. A single balancing loop brings a system stock back to its desired state. Resilience is provided by several such loops, operating through different mechanisms, at different time scales, and with redundancy – one kicking in if another one fails.”

I wonder if we are fostering resilience where it is needed most: military systems. I wonder if putting all our eggs into the F-35 basket provides resilience? How about GPS technologies, the CVN-78 aircraft carrier, the LCS, and so forth? Do we have several operating mechanisms providing feedback loops, or are we looking for the single best platform to fulfill as many missions as possible?

If resilience is achieved through redundancy, and if resilience is a good thing, then an acquisition process built on 19th century ideas of straight-line bureaucracies with zero duplication or overlap may be suboptimal.

Is OSD hydra-headed? Views from 1974

Today's quote and graphic comes from the “Report of the Army Materiel Acquisition Review Committee (AMARC): Volume II Committee Reports.”

"The Army is profoundly affected by OSD leadership and behavior. From the day he is a lieutenant the typical Army officer learn to interpret and accord with duly invested higher authority… Professor Reis (The Management of Defense, John Hopkins Press, 1964) has carefully researched the adversary system between the Services and OSD… The now long drawn out encounter between the two cultures has been detrimental to the Army process of materiel acquisition from a strong but indirect cause...

“OSD is now hydra-headed. Questions pour out of these many heads. The questions can overlap, or deal with the same issues. They appear not to be coordinated at OSD level. The result is tri-service organizational entropy ("an amount of energy in a system not available for doing work") gain.”

The April 1974 report argues that Army staff had grown too large because of OSD intrusion. The report sought to reduce Army Staff, which due to OSD questions was forced to “interdict the materiel acquisition process at all levels and at some length.”

Also, Professor Reis' book cited above is highly recommended.

Below is a nice chart that shows how the Army Program Manager of the Heavy Lift Helicopter had to provide 14 briefings to high level offices in 31 work days (including Christmas-New Year's week) that required five separate trips from St. Louis to D.C.:

Landau on Redundancy

Today's quote is from Martin Landau's classic 1969 essay "Redundancy, Rationality, and the Problem of Duplication and Overlap."

"In complex and tightly ordered systems, the cost of error can run very high...

 “Is it possible to take a set of individually unreliable units and form them into a system “with any arbitrarily high reliability”? Can we, in other words, build an organization that is more reliable than any of its parts?

“The answer, mirabile dictu, is yes. In what is now a truly classical paper, Von Neumann demonstrated that it could be done by adding sufficient redundancy.”

Landau was way ahead of his time on applying complexity theories to organizations. 

Why the program office? Thinking about Organizations

Why does DoD acquisition have a bunch of transient program offices, each of which often has a single prime contractor? What happened to organizations with persistence and culture? I'll provide a speculative taste below that is an excerpt from a much longer paper submitted to ICEAA (I'll post it when they put it up).

... The transfer of weapon systems expenditures to organizations external to the Defense Department brought issues of contracting to the fore. When the actual operations of doing experiments or bending metal occur in-house, the executive may act very much like a military commander in the field. He can express his desires, or lay out his “demand function,” and command action be taken. Depending on how he judges the resulting action when compared with his updated expectations, the executive or commander can reward or punish his subordinates. This method of administrative control is often called after-the-fact control. However, when defense executives seek production from the open market, whether it be firms, universities, or non-profits, they must use market exchange mechanisms characterized by contracts. A contract essentially seeks voluntary agreement between two or more parties where the exact responsibilities of each party and methods for evaluation are detailed before action is taken. A similar method of administrative control is wielded by the controller using program budgets. Both contracts and program budgets use the method of before-the-fact control. Professor Thompson compared the two methods with respect to internal administration:

“[Before-the-fact] controls necessarily take the form of authoritative mandates, rules, or regulations that specify what the subject must do, may do, or must not do. The subjects of before-the-fact controls are held responsible for complying with these commands and the controller attempts to monitor and enforce compliance with them.

“After-the-fact controls are executed after the subject, either an organization or an individual, decides on and carries out a course of action and, therefore, after some of the consequences of the subject's decisions are known.”[1]

The congruence between the methods of contracting and program budgeting made the two natural bedfellows, the enabler being the unifunctional project office structure. The program budget demands that organizations find perfect alignment with program structure, which Mosher had showed impossible for any significant organization. These forces, coupled with an expressed desire for private production, led the Air Force to favor the systems project office and a single prime contractor. For the pre-existing Army and Navy organizations to be viable, there needed to exist an auditable accrual accounting system by program. Without such an accounting system, the policy maker could not effectively monitor execution to plan; nor could the policy maker forecast future plans. Though Congress mandated such an accrual accounting system in 1955, it was never accomplished.[2] Thus, the move toward unifunctional project offices can be seen as a means of outsourcing accounting compliance as well as production knowledge to industry. Control through the program budget would otherwise require multifunctional in-house organizations to perform such accounting themselves.

---

[1] Thompson, Fred. “Public Economics and Public Administration.”

http://www.willamette.edu/~fthompso/ECON&PA.html.

[2] Lander, Ezra. “Performance Budgeting and Accounting Policy in the Department of the Army.” The American University, Ph.D., 1961, pp. 242. Accrual accounting was first emphasized to the Congress in the Second Hoover Commission of 1955, but its need was understood much earlier. Despite repeated mandates, an auditable accrual accounting system remains years away in the Department of Defense.

Supercomputer to navigate regulations?

Here is Scott Chandler in a Lexington Institute article:

"The defense acquisition system is a construct of government, erected over decades and codified in statute that now exceeds 180,000 pages.  It is so complex that the Air Force commissioned a supercomputer to make sense of it."

I'll leave the 180,000 pages alone. The news article announcing the Air Force supercomputer to deal with the acquisition process should give one pause. Here is the Air Force justification:

"The Air Force pointed to a 2006 study by the Government Accountability Office, Congress’ top watchdog, which found that 'the challenge of operating in accordance with complex federal acquisition regulations discourages small and innovative businesses from partnering with the government in emerging markets.'

"The Air Force hopes the new system will be an advanced tool, making it easier for businesses to understand the requirements of a contract and to get any of their questions answered immediately by the computer system."

So to encourage small business and innovation in new markets... the Air Force wants a supercomputer to navigate the deluge of regulations? It's not clear what processes the supercomputer would execute, how all the rules will be simplified through number crunching, what information people could to "mine," and whether its biggest benefit accrue to existing prime contractors and lead to further consolidation.

A pristine farm of super computers

Here is a thought to chew on: the Air Force will "teach the system how to understand context so that it can answer questions accurately."

Technology readiness and the F-35

The Straus Military Reform Project has a new set of podcasts taking the name of their nice 2011 book, The Pentagon Labyrinth. One episode by Dan Grazier, "The F-35 continues to stumble," provides a good exposition of the F-35's current status. There are so many points, but here is one representative example:

"The cannon in the F-35A sits behind a small door on the side of the aircraft that opens quickly an instant before the cannon is fired—a characteristic intended to keep the aircraft stealthy. Test flights have shown that this door catches the air flowing across the surface of the aircraft, pulling the F-35’s nose off the aimpoint resulting in errors “that exceed accuracy specifications.”"

Read the whole article if you don't listen to the podcast. I won't go on here about well documented troubles of the F-35. But the gun issue reminded me of another matter.

I had recently been reading about what people said about the F-35 program back when it was going through milestone B in 2002. The hot topic of the day was "knowledge-based acquisition," where information on technological readiness drives the progression of program phases. The Technology Readiness Level (TRL) was developed as an industry best practice for measuring knowledge achieved. For example, programs should not enter full scale development (pass Milestone B) until critical technologies reach a TRL of at least 6, where system/subsystems have been modeled or prototyped in a realistic environment.

What's the point? Certainly no one was debating the technical readiness of the F-35 gun, and even if they did, it would be at the highest level of 9. But was anyone thinking about interactions between technologies, and how they would operate in their actual environment? Time and again it is not until a system or subsystem is put to the test that critical problems are exposed, and all our supposed knowledge implied by the TRL means little or nothing.

Cost Growth, a brief interpretation

Today’s quote is from page 273 of the Second Hoover Commission Hearings (1957):

“Mr. Brown. [The Department of Defense] seldom get anything for the price tag originally put on it…

“Mr. McNeil. That was certainly true some years ago… For about 4 years now, most estimates are on the beam. And it was always possible.”

It is hard to believe that between 1953 and 1957 defense cost estimates had been “on the beam,” particularly when Martin Peck and Frederic Scherer found substantial cost growth in the 1950s in their classic The Weapons Acquisition Process: An Economic Analysis. Maybe they were just better than the estimates developed during the Korean War crisis.

Cost estimation continues to be a major problem today. In fact, it is widely agreed that cost growth has come down relative to the 1950s period. I doubt any informed observer would say we have contained cost growth, or that we are doing better today than in the 1950s, despite the fact that the AT&L’s “Performance of the Defense Acquisition System” repeatedly trumpets cost containment.

It may not take a searching review to understand that cost growth need not have any relationship with program outcomes. As Robert Devons wrote, cost growth studies “showed that most of the so called cost growth was the direct result of low initial cost estimates.”

So, as Donald Srull said, “One could not after all, buy the new Cadillac automobile for $5,000. When it turned out to cost $25,000 it was not “cost growth”—it was an unrealistic, poor initial underestimate!” (Srull was the first chairman of the Cost Analysis Improvement Group, or CAIG.)

Our problem today is that in a decision maker’s mind, a higher cost estimate is a more “realistic” one. If a realistic cost estimate for a $25,000 Cadillac comes out to be $40,000, anything other than large cost underruns signal escalating backend prices. Unfortunately, “should cost” initiatives cannot deliver the required information. We only know a $25,000 Cadillac doesn’t cost $40,000 because there is a market for Cadillacs which is embedded in an even broader market framework. You cannot just add up the labor and materials required to produce technological innovation.

The problem more pernicious than cost growth is simply high costs (similar to the “cost disease” inflicting the commanding heights of education and healthcare). It might be stunning to some that the Bruce Harmon of the Institute for Defense Analysis found that the fighter aircraft industry experiences price escalation at nearly twice the rate seen in the economy at large. (In other words, for a constant quality fighter aircraft our “buying power” is falling by about 2%-3% per year—every couple decades we can only afford half as much with the same inflation-adjusted dollars.)

Harmon’s conclusion… the F-35 is an average performing program and it experienced “cost growth” because of its artificially low Milestone B cost estimate that did not assume fast enough price growth in line with experience! If the F-35 is not really a cost growth problem, then cost growth is not the problem we should be looking at.