Why the program office? Thinking about Organizations

Why does DoD acquisition have a bunch of transient program offices, each of which often has a single prime contractor? What happened to organizations with persistence and culture? I'll provide a speculative taste below that is an excerpt from a much longer paper submitted to ICEAA (I'll post it when they put it up).

... The transfer of weapon systems expenditures to organizations external to the Defense Department brought issues of contracting to the fore. When the actual operations of doing experiments or bending metal occur in-house, the executive may act very much like a military commander in the field. He can express his desires, or lay out his “demand function,” and command action be taken. Depending on how he judges the resulting action when compared with his updated expectations, the executive or commander can reward or punish his subordinates. This method of administrative control is often called after-the-fact control. However, when defense executives seek production from the open market, whether it be firms, universities, or non-profits, they must use market exchange mechanisms characterized by contracts. A contract essentially seeks voluntary agreement between two or more parties where the exact responsibilities of each party and methods for evaluation are detailed before action is taken. A similar method of administrative control is wielded by the controller using program budgets. Both contracts and program budgets use the method of before-the-fact control. Professor Thompson compared the two methods with respect to internal administration:

“[Before-the-fact] controls necessarily take the form of authoritative mandates, rules, or regulations that specify what the subject must do, may do, or must not do. The subjects of before-the-fact controls are held responsible for complying with these commands and the controller attempts to monitor and enforce compliance with them.

“After-the-fact controls are executed after the subject, either an organization or an individual, decides on and carries out a course of action and, therefore, after some of the consequences of the subject's decisions are known.”[1]

The congruence between the methods of contracting and program budgeting made the two natural bedfellows, the enabler being the unifunctional project office structure. The program budget demands that organizations find perfect alignment with program structure, which Mosher had showed impossible for any significant organization. These forces, coupled with an expressed desire for private production, led the Air Force to favor the systems project office and a single prime contractor. For the pre-existing Army and Navy organizations to be viable, there needed to exist an auditable accrual accounting system by program. Without such an accounting system, the policy maker could not effectively monitor execution to plan; nor could the policy maker forecast future plans. Though Congress mandated such an accrual accounting system in 1955, it was never accomplished.[2] Thus, the move toward unifunctional project offices can be seen as a means of outsourcing accounting compliance as well as production knowledge to industry. Control through the program budget would otherwise require multifunctional in-house organizations to perform such accounting themselves.

---

[1] Thompson, Fred. “Public Economics and Public Administration.”

http://www.willamette.edu/~fthompso/ECON&PA.html.

[2] Lander, Ezra. “Performance Budgeting and Accounting Policy in the Department of the Army.” The American University, Ph.D., 1961, pp. 242. Accrual accounting was first emphasized to the Congress in the Second Hoover Commission of 1955, but its need was understood much earlier. Despite repeated mandates, an auditable accrual accounting system remains years away in the Department of Defense.

Supercomputer to navigate regulations?

Here is Scott Chandler in a Lexington Institute article:

"The defense acquisition system is a construct of government, erected over decades and codified in statute that now exceeds 180,000 pages.  It is so complex that the Air Force commissioned a supercomputer to make sense of it."

I'll leave the 180,000 pages alone. The news article announcing the Air Force supercomputer to deal with the acquisition process should give one pause. Here is the Air Force justification:

"The Air Force pointed to a 2006 study by the Government Accountability Office, Congress’ top watchdog, which found that 'the challenge of operating in accordance with complex federal acquisition regulations discourages small and innovative businesses from partnering with the government in emerging markets.'

"The Air Force hopes the new system will be an advanced tool, making it easier for businesses to understand the requirements of a contract and to get any of their questions answered immediately by the computer system."

So to encourage small business and innovation in new markets... the Air Force wants a supercomputer to navigate the deluge of regulations? It's not clear what processes the supercomputer would execute, how all the rules will be simplified through number crunching, what information people could to "mine," and whether its biggest benefit accrue to existing prime contractors and lead to further consolidation.

A pristine farm of super computers

Here is a thought to chew on: the Air Force will "teach the system how to understand context so that it can answer questions accurately."

Technology readiness and the F-35

The Straus Military Reform Project has a new set of podcasts taking the name of their nice 2011 book, The Pentagon Labyrinth. One episode by Dan Grazier, "The F-35 continues to stumble," provides a good exposition of the F-35's current status. There are so many points, but here is one representative example:

"The cannon in the F-35A sits behind a small door on the side of the aircraft that opens quickly an instant before the cannon is fired—a characteristic intended to keep the aircraft stealthy. Test flights have shown that this door catches the air flowing across the surface of the aircraft, pulling the F-35’s nose off the aimpoint resulting in errors “that exceed accuracy specifications.”"

Read the whole article if you don't listen to the podcast. I won't go on here about well documented troubles of the F-35. But the gun issue reminded me of another matter.

I had recently been reading about what people said about the F-35 program back when it was going through milestone B in 2002. The hot topic of the day was "knowledge-based acquisition," where information on technological readiness drives the progression of program phases. The Technology Readiness Level (TRL) was developed as an industry best practice for measuring knowledge achieved. For example, programs should not enter full scale development (pass Milestone B) until critical technologies reach a TRL of at least 6, where system/subsystems have been modeled or prototyped in a realistic environment.

What's the point? Certainly no one was debating the technical readiness of the F-35 gun, and even if they did, it would be at the highest level of 9. But was anyone thinking about interactions between technologies, and how they would operate in their actual environment? Time and again it is not until a system or subsystem is put to the test that critical problems are exposed, and all our supposed knowledge implied by the TRL means little or nothing.

Cost Growth, a brief interpretation

Today’s quote is from page 273 of the Second Hoover Commission Hearings (1957):

“Mr. Brown. [The Department of Defense] seldom get anything for the price tag originally put on it…

“Mr. McNeil. That was certainly true some years ago… For about 4 years now, most estimates are on the beam. And it was always possible.”

It is hard to believe that between 1953 and 1957 defense cost estimates had been “on the beam,” particularly when Martin Peck and Frederic Scherer found substantial cost growth in the 1950s in their classic The Weapons Acquisition Process: An Economic Analysis. Maybe they were just better than the estimates developed during the Korean War crisis.

Cost estimation continues to be a major problem today. In fact, it is widely agreed that cost growth has come down relative to the 1950s period. I doubt any informed observer would say we have contained cost growth, or that we are doing better today than in the 1950s, despite the fact that the AT&L’s “Performance of the Defense Acquisition System” repeatedly trumpets cost containment.

It may not take a searching review to understand that cost growth need not have any relationship with program outcomes. As Robert Devons wrote, cost growth studies “showed that most of the so called cost growth was the direct result of low initial cost estimates.”

So, as Donald Srull said, “One could not after all, buy the new Cadillac automobile for $5,000. When it turned out to cost $25,000 it was not “cost growth”—it was an unrealistic, poor initial underestimate!” (Srull was the first chairman of the Cost Analysis Improvement Group, or CAIG.)

Our problem today is that in a decision maker’s mind, a higher cost estimate is a more “realistic” one. If a realistic cost estimate for a $25,000 Cadillac comes out to be $40,000, anything other than large cost underruns signal escalating backend prices. Unfortunately, “should cost” initiatives cannot deliver the required information. We only know a $25,000 Cadillac doesn’t cost $40,000 because there is a market for Cadillacs which is embedded in an even broader market framework. You cannot just add up the labor and materials required to produce technological innovation.

The problem more pernicious than cost growth is simply high costs (similar to the “cost disease” inflicting the commanding heights of education and healthcare). It might be stunning to some that the Bruce Harmon of the Institute for Defense Analysis found that the fighter aircraft industry experiences price escalation at nearly twice the rate seen in the economy at large. (In other words, for a constant quality fighter aircraft our “buying power” is falling by about 2%-3% per year—every couple decades we can only afford half as much with the same inflation-adjusted dollars.)

Harmon’s conclusion… the F-35 is an average performing program and it experienced “cost growth” because of its artificially low Milestone B cost estimate that did not assume fast enough price growth in line with experience! If the F-35 is not really a cost growth problem, then cost growth is not the problem we should be looking at.

Economics merely a branch of decision theory?

Bonus quote today is found on page 510 of Philosophy of Complex Systems, from an essay by John Foster called "Economic Systems" (2011):

"Over the past half century, economic theorists have stubbornly held on to their view that economics should be a branch of decision theory that involves optimizing choices along artificially smooth and conveniently specified production and utility functions, subject to constraints."

A lot was said in that sentence. Economists were "stubborn" because, despite the continual failure of their models to describe reality, they hold on to assumptions of "artificially smooth" and "conveniently specified" functions. They hold onto these methods because linearity is mathematically tractable (the same critique is sometimes levied on physics).

But clearly we live in a complex world dominated by numerous changing factors. For this reason, it is misleading to think of economics as "a branch of decision theory that involves optimizing choices" and instead one should think about how individuals interact and exchange.

As Armen Alchian concluded in 1954:

"Systems analyses are machines for generating implications of postulated initial information; they do not generate decisions...  Under uncertainty, the criterion of decisions is not simple maximizations; the essence of the decision process is to affect the scope of random factors so as to give a “good” probability distribution of outcomes. The insurance principle is to decisions what maximizations are to analytic implications."

His 1950 paper "Uncertainty, Evolution, and Economy Theory" argued that optimal decisions can only be regarded as such after-the-fact through a process of natural selection. Decision problems under uncertainty were solved, he later found, by maximizing the option space available. He agreed with Herbert Simon that one should optimize when one has the necessary information, but an evolutionary "satisficing" method probably more appropriate given the vagaries of our world.

Industrial Consolidation, Aircraft Series

      Today's quote is from "An Initial Look at Technology and Institutions on Defense Industry Consolidation."

     ".... nine prime contractors designed and flew 40 different fighter aircraft designs during the 1940s and 1950s (Lorell & Levaux, 1998)... However, as jet fighter technology matured the number of aircraft designs fell with less than a dozen U.S. fighter aircraft developed since 1960...  

     "A related observation that supports this statement is that there have been no new entrants into U.S. manned aircraft production since 1945 (Birkler, Bower, Drezner, Lee, Lorell, Smith, Trimble, &Younossi, 2003).”

Nassim Taleb on forecasting

The bonus quote today is from Nassim Taleb's The Black Swan, on page 163: 

"There are those people who produce forecasts uncritically. When asked why they forecast, they answer, “Well, that’s what we’re paid to do here.” My suggestion: get another job."

I hear a lot about predictive analytics and the like, but it's not clear that new fads made possible by big data and machine learning are more useful to complex organizations than earlier techniques. Most perniciously, attempts to supply the data and make use of the analysis injects so much rigidity that organizations are robbed of their vitality and initiative.

Edward Luttwak on Efficiency and War

Today's quote is from pages 136-38 of Edward Luttwak's The Pentagon and the Art of War: The Question of Military Reform, 1985.

“The divide between efficiency and effectiveness is at its deepest in combat. What are the most effective military operations? In very general terms, they are operations in which the enemy is not outfought by greater firepower, greater numbers, greater bravery, and greater sacrifice of lives, but instead surprised and then outmaneuvered and disrupted, and thus never given a chance to fully employ his fighting capacity in the first place.

“And what is the common denominator of such operations? Inefficiency 

“Consider surprise. How is it achieved? By deception – unless the enemy is merely apathetic or unobservant, and therefore outclassed to begin with. And how is deception achieved? By doing the unexpected. And what is the unexpected? Something other than the sensible, normal, and efficient."

Luttwak connects the strategic concept of inefficiency to weapon system acquisition:

"But conflict is not like civilian business and efficiency is the wrong goal to pursue: efficiency in making a radar or refueling a ship, of course; efficiency in making radars, or refueling ships, no, for efficient economies of scale in purchasing radars lead to a single mass-produced radar that will be more easily counter-measured, and efficient refueling leads to a few large fleet oilers that are more easily intercepted and destroyed by the enemy. (Each of our majestic aircraft-carrier task forces is now dangerously dependent on a single, very large, very efficient resupply ship.) Conflict is different.” 

The concepts do not just apply to conflict, but to all dynamic systems. Certainly the market economy, itself a dynamic system, fosters inefficiency due to competition and redundancy. Like handling the uncertainties of war, perhaps local inefficiencies produce overall effectiveness of the market system to adapt and progress.

Today's Quote

Is from pages 273-78 of James E. Hewes, Jr.'s "From Root to McNamara: Army Organization and Administration" http://www.history.army.mil/books/root/

"Technical service chiefs could and did transfer funds freely among their various activities, functions, and installations, but neither the Secretary nor the General Staff could legally transfer funds among the several technical services or other staff agencies without going to Congress for approval."

That was before passage of the 1949 amendment to the National Security Act. Title IV instituted the first "performance budget," or program budget that is the prototype of the Planning-Programming-Budgeting-Execution System (PPBES) in use today. It was the brainchild of Ferdinand Eberstadt and Wilfred J. McNeil, the latter becoming Assistant Secretary of Defense(Comptroller), for the next decade. 

Hewes goes on:

"With this one directive McNeil wiped out the independent budgets of the technical services dating back in some instances to the Revolution. The chiefs no longer would defend their budgets before Congress. Instead this would be the responsibility of the several General Staff divisions. Congressional restrictions on transferring funds among appropriations would hamstring the technical services rather than the General Staff."

Before 1949, the budget was controlled by independent heads of different Army organizations, providing competition and diversity -- particularly in the technical services which performed weapons procurement. Afterwards, it was to be organized centrally from Army staff under the authority of the Secretary of Defense and his Comptroller.

Perhaps the most pernicious part of it all, the performance budget controlled not just the flow of dollars, but the precise objectives those dollars were to accomplish. No longer could a project manager decide to take alternative approaches to new technologies on his own authority, and contract freely with industry.