We have moved

Blog has moved to AcquisitionTalk.com.

The F/A-18 that almost wasn't

Today I'd like to discuss how fragile the defense acquisition system is by explaining how the Navy came very close to never getting it's air combat mainstay, the F/A-18.

As is well known, the Fighter Mafia of the 1960s advocated simple aircraft designs based on Capt. John C. Boyd's energy manuverability (EM) theory. This resulted in the F-15 aircraft that dug the Air Force out of a bad pickle with the poor performing F-111 aircraft.

However, as the F-15 weight and cost grew, the Air Force starting looking at a lightweight fighter. In keeping with Deputy Secretary of Defense's prototyping methodology, the Air Force funded a fly-off between the General Dynamics YF-16 and Northrop's YF-17. Only a last minute deal made this happen.

As is well known, the YF-16 won the Air Force fly-off.

The Congress intended the Navy to develop a derivative from the YF-16 to increase commonality and reduce overall costs. However, when the Navy did a paper design competition, it found the YF-16 derivatives (1600, 1601, and 1602) were inadequate. The Navy selected a derivative from the Air Force loser, the YF-17. It became called the F/A-18.

The Navy award to Northrop, potentially a mending process aftger Northrop became persona non grata with the DoD due to the F-14 failures, in direct defiance of the Congressional conference report. Naturally, a contract protest was filed to stop the contract award that defied the purpose of the funds disbursed.

YF-16 (near) and YF-17 (far)

The Congress held several meetings during 1975 to investigate the Navy acquisition. The clear tenor of the Congressmen was outrage at the Navy's actions -- first for defying the law; and second for generating more costs due to a duplicative program without having performed the necessary cost-effectiveness analyses. Senator Barry Goldwater said on September 17, 1975:

"So, Mr. Chairman, we may have missed the purpose of the whole exercise which, I htought, was the development of a single effective fighter aircraft for both services that could be afaforded in the numbers required and with the corresponding high degree of commonality, life cycle costs would be reduced significantly. This may only be an impossible dream that some of us have, bbut I believe our services must figure out some way to get together on many of these aircraft systems because we cannot continue forever to pay for these separate air forces with their completely different inventories."

The Navy pointed out that making the YF-16 carrier suitable required extensive modifications. For example, The YF-16 had an empty weight of 13.6K lbs, whereas its carrier derivative increased to 18.6K lbs. Wing span increased from 28 to 33 feet, and the horizontal tail area increased from 42 to 75 squared-feet. By contrast, the F/A-18 increased the empty weight of the YF-17 from 17K lbs to 20.6K lbs. It also had an improved General Electric F-404 engine with 5-15% more trust.

It wasn't until the GAO report of October 10, 1975, that minds started changing. The GAO found that the Navy award to Northrop did not contradict the law,  because the Congressional conference report on the allocation of funds was not legally binding. But the GAO remained neutral on the selection.

The funding for the F/A-18 almost didn't make it through the House, but as Kelly Orr found in his history "Hornet,"  an "odd coalition" of pro defense conservative and liberals from the states like California, Massachusetts, which was home to companies like Northrop and General Electric, voted it through. The vote made it despite opposition from Chairman Mahon of the appropriations commmittee, who was from Texas, home to General Dynamics and LTV.

LTV actually paired with General Dynamics because it was convinced winning the Air Force competition would assure its victory in the Navy compete. LTV even turned away a superior deal from Northrop.

While the House vote nearly assured the F/A-18's approval, Senators remained displeased. Goldwater, to some extent, reversed his opinion. He realized that the economies-of-scale provided by a common Air Force/Navy program would not materialize because of the vast difference in mission requirements. So he stated that:

"... I don't believe, in my years in the Congress, I have ever opposed a weapon system. I want to make it clear that I don't oppose the F-18 weapon system. I oppose the way that they have gone about obtaining it."

It would appear that Goldwater's earlier statements, where he opposed the program on principle that it was not cost-effective, miraculously changed. It was just that the Navy didn't make a good cost-effectiveness case.

Had the GAO ruled against the Navy, I could not see the F/A-18 having ever been developed. It would probably have been either a derivative of the F-14, as pushed earlier, or a derivative of the F-16.

Without highly improbable confluence of John C. Boyd's genius (see Osinga, "Science, Strategy, and War") and David Packard's prototyping reform initiatives, there probably would never have been a F/A-18, nor would there have been an F-16.

Any and all sound attempts to promote diversification in DoD acquisitions are inherently fragile due to the misguided ideals of zero redundancy inherent to the PPBS way of thinking. It forces a static cost-effectiveness mentality that muddies the issues and subtlely injects biases. This is the hoolahoop all programs must go through to get at funding in the PPBS -- a convincing cost-effectiveness analysis.

At the time, prototyping competitions cost in the order of tens of millions of dollars, while full procurement (not including operating/support) went into the billions, if not tens of billions. Prototypes -- and full-scale development -- are learning processes. One cannot fully justify the end-goal of a prototype, it is to learn what is possible and what you might want.

Certainly the DoD can afford more prototypes, and even more full-scale developments. Certainly the F-16 and F/A-18, despite their apparent duplication, broke the trend of exponentially increasing costs and limited performance gains (though, with the F-35 we are back on that trend). And certainly, the Navy should have gone through its own fly-off before buying -- but who can blame them given the precariousness of their funding position with the Congress.

Rickover and the "Nickle Letter"

Today's quote comes from page 53-54 of the 1968 Senate hearings on Economy in Military Procurement, Part 2, featuring Admiral Hyman G. Rickover.

"Admiral Rickover. I reccall a recent experience with this type of thinking "the nickle letters." In August of this year the Navy proposed to place a $50 million contract with a company at a profit of 2.29 percent.

"Chairman Proxmire. Let me understand the 2.29 percent figure. Was that the percentage of profit to sales or to cost?

"Admiral Rickover. It is 2.29 percent of estimated cost. That may sound like a low profit -- 

"Chairman Proxmire. It does indeed. In testimony yersterday, the Department of Defense witness said that the average profit on defense work was 9.4 percent.

"Admiral Rickover. Actually, it was quite adequate under the circumstances. The contract incolved no risk for the company and almost no investment, and the Navy has been working on the same terms with this company for many years.

"In any event, because of the amount oof this contract, it had to be approved by higher authority. When I submitted the contract for approval, I received a formal letter stating the contract was disapproved because the profit was too low....However, in order to have the contract approved, I was willing to increase the fee on this $50 million contract from $1,147,023 to 1,147,023.05 -- and increase from 2.29 percent to 2.29000001  percent."

The nickle letter from Rickover shows the DoD's obsession with profit because it have no understanding of value. Rickover quoted Oliver Cromwell who said: "I beeseach you, in the bowels of Christ, to think it possible you have been mistaken." He had to fight the Navy to let a contract that the firm already agreed to!

Today, there does not seem to be the exact same issue. For example, in billion dollar service contracts where there is little or no risk, there is no question the government will pay a small fee in percentage terms, but a huge one in dollar terms.

As Rickover later showed, almost all of the contract to the prime would be subcontracted out. The prime would only incur $1.473 million in labor costs and receive $1.147 million in profit. Of course, the prime's overhead costs outweighed its labor costs. But that's closer to 50% profit on capital invested. Not too shabby.

The general popint is that the meaningful measure of profitability is the percentage profit on invested capital, and not total sales. Retailers such as Amazon or Wal Mart may make far less profit in percentage terms to their total sales because they are not as "vertically integrated" as, say, commodity producers are. They contribute little end value to the item, so their profits should be lower.

This outcome is natural in market competition, but needs to be willfully imposed in the non-market environment of defense procurement.

It appears forgotten in Rickover's biographies just how sophisticated he was in contracting and procurement. In this hearing, Rickover had no preparation (because he was not initially aware they were occuring) and he still provided an absurdly detailed discussion on a vast array of issues.

Admiral Rickover on improving weapons acquisition

Today's quote is from Admiral Hyman G. Rickover's memo to David Packard on May 22, 1970, concerning Packard's 5000 series directives to improve the acquisition process:

“... to bring about real improvement in weapons acquisition cannot be corrected by management policy directives… My experience has been that when a directive such as the one you propose is issued, most of the effort goes into the creation of additional management systems and reports and the preparation of large numbers of documents within the Service to “prove” that the requirements of the directive are being me—in order to justify funds for the Service…. I think that if you check you will find that virtually all of these documents, which have required so much effort to prepare, are used almost entirely to gain approval for programs, and are simply filed away once funds are released. They are of no use to the working people in performing their jobs. In my opinion the present situation in this regard is worse than I have ever seen it. Further, I predict that the way the present bureaucracy would implement your proposed directive will make matters still worse.”

Rickover was prescient on this matter; inevitably any streamlining and decentralizing initiative gets drowned out by the bureaucratic structure. Yet perhaps that is for good reason, because true deregulation simply cannot happen in today's industrial culture due to the wastefulness and deception contractors can get away with.

In some ways, the statement "trust, but verify" makes a lot of sense in defense acquisition. There must be a building of trust between a multiplicity of actors, not just a monolithic government and a handful of firms. But for trust to solidify, there needs to be verification of that trust -- namely understanding the value of project results.

If the purchaser can properly evaluate a project outcome, which only comes after years of experience with numerous similar projects, then trust is a matter of after-the-fact evaluation followed by reward or punishment (generally filtered through reputation effects, not legal recourse).

If the purchaser knows relatively little about evaluation, evident by constant information gathering and justification requirements, then trust may appear to be a matter of ensuring proper accounting and regulation of profit levels.

The problem with this story, however, is that no one was better than Rickover at understanding technical aspects of engineering projects, and Rickover still relied on cost accounting standards to reveal fraud or abuse.

Though fraud and abuse certainly did occur, it was because of Rickover's other complaint, that the lower echelons were losing technical talent fast and there wasn't a cultural knowledge of project evaluation, and more importantly, they couldn't make project decisions without getting funding requests through 30 or more offices at all levels of the Department. 

Nothing more than this can stamp out the diversity of organizations and projects leading to what is most important: the increased information processing capacity of the Defense system devoted to "doers" and not "reviewers," as Rickover would say.

Roots of the PPBS and scientific mgmt

The Planning-Programming-Budget System (PPBS) found its roots squarely with RAND, but reflected a broader trend in public administration dating back in the U.S. to the late nineteenth century. 

If the logic of military unification derived from German concepts of administration and the general staff, then the PPBS derived from the German historical school of economics.[1] Essential to the German tradition is analytical holism and a rejection of the “fictitious individualistic assumption” of classical liberals. Because markets produced social and economic failures, particularly monopoly, a new class of expert were required to identify remedies using the administrative state. 

The economist as an American profession was built on men schooled in Germany, who then solidified their expertise by creating university departments, prestigious associations, and new government bureaus on statistics and regulation. 

To justify its role for guiding government, the economic expert relied on the legitimacy of the scientific method. One top expert, Henry Farnam, compared the evolution of the economic sciences to the medical sciences. He found that surgery was once primitive and dangerous, but advances in science had made it most beneficial to society. Similarly, the economic expert had by 1910 enough scientific knowledge to make its reforms “more effective and less dangerous.”[2] 

The analogy was repeated over 50 years later by Alain C. Enthoven, Assistant Secretary of Defense for Systems Analysis, who said “My general impression is that the art of systems analysis is in about the same stage now as medicine during the latter half of the 19^th century; that is, it has reached the point at which it can do more good than harm.”[3]

Alain C. Enthoven was 31 when he became head of the
Office of Systems Analysis

---

[1] F. A. Hayek traced the ideas inherent in the German school, particularly logical positivism, back to the French Revolutionaries and the Ecole Polytechnique – but he attributed its spread to the U.K. and U.S. through Germany. This form of rationalism also finds precedent thinkers such as Francis Bacon, who opposed Copernican Astronomy, and Lord Kelvin, who denied evolution because he calculated the Earth too young for its emergence. Ultimately analytical holism goes back to Plato, who believed in a “Guardian” class to guide policy and abhorred asymmetries so much that he thought humans should use both hands with equal dexterity.

[2] Leonard, Thomas C. Illiberal reformers, Princeton University Press, 2016, pp. 22, 33

[3] “Planning Programming Budgeting” Inquiry of the Subcommittee on National Security and International Operations (Jackson Committee), U.S. Senate, U.S. Government Printing Office, Washington: 1970, pp. 127

Why not fixed budgets?

Today's quote is from page 239 of the 1960 book, The Economics of Defense in the Nuclear Age:

Charles Hitch

“With existing institutions, for example, the bargainers are often given the perverse incentive of trying to maximize their budgets (their costs) instead of maximizing capability for whatever budget they receive. A subordinate official may well feel that effort devoted to getting his budget increased is more rewarding than analysis and effort devoted to getting greater capability from a given budget.

“This tendency might be countered by some device for permitting units to keep a part of any cost savings. An extreme form of correction, which admittedly has certain disadvantages, would be to give each unit a budget fixed in amount for two (or even more) years in advance.”

Here, the principal founders (Charles Hitch and Roland McKean) of modern defense resource management admit that there was some logic to the preexisting system of very broad project discretion within a fixed budget, as opposed to make a case for as big of a budget as possible based on the requirements involved.

Why not provide fixed budgets?

Advanced prototyping, a last minute deal for the F-16

Today's quote is from the 1971 Senate hearings on Advanced Prototyping. Here is Air Force General K. R. Chapman:

"... we think three- to five-man teams on each project working closely with the contractor, strongly supported by our inhouse laboratories and systems division, would compare very favorably with 50- to 250-man program offices that are in our full scale production efforts.

“Some of the principles that we have looked at, which we think are useful here, are to reduce the requirement for special reporting by the use of the contractor formatted data; to waive or set aside several hundred procedural policy regulations, manuals and directives that normally govern our full development, procedure; also to, in other cases, selectively apply but not contractually invoke some of these existing directives. Reporting of the program managers would be kept as simple and direct as possible and in terms of the split test program we are talking about the services and the contractor jointly performing this with the contractor retaining the right through his designers to make changes during that program...

"We believe that our solicitation can be reduced to a Request for Proposal of about 25 pages compared to what we are doing today. We think the contractor response can be held to about 60 pages compared to what we have received today.”

General Chapman was one of the service representatives accompanying Deputy Secretary of Defense David Packard. They came to the Senate on September 9, 1971 to plead for additional prototype funding in the FY1972 budget, which officially began on October 1, 1971. Further, Packard said "We believe this should be an authorization rather than a reprograming or tradeoff action."

In other words, increase the DoD top line with three weeks left to the start of budget execution. Rather unorthodox, one might suppose. The amount came to $63.7 million. I wonder if some of that money went directly to the YF-16 and YF-17 flyoff:

"In January 1972, proposals were sought for a fighter with excellent acceleration, turn rate and range in the 20,000-pound weight class… In February 1972, Lockheed, General Dynamics, Boeing, Northrop and LingTemco-Vought (LTV, later Vought) submitted proposals… The Air Force selected General Dynamics and Northrop in April 1972 to design and build two prototypes each"

Notice how they took no time at all, requesting proposals from industry just four months after begging for the money and receiving four bids just a month after that; it only took another two months to select the winners. Both were truly winners, as General Dynamics would go on to build the F-16 after winning the Air Force competition and Northrop the F-18, which evolved out of the YF-17 that lost the competition. Both are mainstays.

I'll frame this as a questionable last minute budget deal, purposefully put out in front of the public, that had unquestionable good results. However favorable prototyping is, and it is favorable, the situation feels like short-termism that allowed the DoD not to internalize the reform it asked for. Packard himself resigned two months after the Sepember 1971 hearings.

That second quote was from “Quest to Build a Better Fighter” by Michael Sanibel.

How do we create military resilience?

Today's quote is from page 76 of Donella Meadow's Thinking In Systems:

 "Resilience arises from a rich structure of many feedback loops that can work in different ways to restore a system even after a large perturbation. A single balancing loop brings a system stock back to its desired state. Resilience is provided by several such loops, operating through different mechanisms, at different time scales, and with redundancy – one kicking in if another one fails.”

I wonder if we are fostering resilience where it is needed most: military systems. I wonder if putting all our eggs into the F-35 basket provides resilience? How about GPS technologies, the CVN-78 aircraft carrier, the LCS, and so forth? Do we have several operating mechanisms providing feedback loops, or are we looking for the single best platform to fulfill as many missions as possible?

If resilience is achieved through redundancy, and if resilience is a good thing, then an acquisition process built on 19th century ideas of straight-line bureaucracies with zero duplication or overlap may be suboptimal.

Is OSD hydra-headed? Views from 1974

Today's quote and graphic comes from the “Report of the Army Materiel Acquisition Review Committee (AMARC): Volume II Committee Reports.”

"The Army is profoundly affected by OSD leadership and behavior. From the day he is a lieutenant the typical Army officer learn to interpret and accord with duly invested higher authority… Professor Reis (The Management of Defense, John Hopkins Press, 1964) has carefully researched the adversary system between the Services and OSD… The now long drawn out encounter between the two cultures has been detrimental to the Army process of materiel acquisition from a strong but indirect cause...

“OSD is now hydra-headed. Questions pour out of these many heads. The questions can overlap, or deal with the same issues. They appear not to be coordinated at OSD level. The result is tri-service organizational entropy ("an amount of energy in a system not available for doing work") gain.”

The April 1974 report argues that Army staff had grown too large because of OSD intrusion. The report sought to reduce Army Staff, which due to OSD questions was forced to “interdict the materiel acquisition process at all levels and at some length.”

Also, Professor Reis' book cited above is highly recommended.

Below is a nice chart that shows how the Army Program Manager of the Heavy Lift Helicopter had to provide 14 briefings to high level offices in 31 work days (including Christmas-New Year's week) that required five separate trips from St. Louis to D.C.:

Landau on Redundancy

Today's quote is from Martin Landau's classic 1969 essay "Redundancy, Rationality, and the Problem of Duplication and Overlap."

"In complex and tightly ordered systems, the cost of error can run very high...

 “Is it possible to take a set of individually unreliable units and form them into a system “with any arbitrarily high reliability”? Can we, in other words, build an organization that is more reliable than any of its parts?

“The answer, mirabile dictu, is yes. In what is now a truly classical paper, Von Neumann demonstrated that it could be done by adding sufficient redundancy.”

Landau was way ahead of his time on applying complexity theories to organizations. 

Why the program office? Thinking about Organizations

Why does DoD acquisition have a bunch of transient program offices, each of which often has a single prime contractor? What happened to organizations with persistence and culture? I'll provide a speculative taste below that is an excerpt from a much longer paper submitted to ICEAA (I'll post it when they put it up).

... The transfer of weapon systems expenditures to organizations external to the Defense Department brought issues of contracting to the fore. When the actual operations of doing experiments or bending metal occur in-house, the executive may act very much like a military commander in the field. He can express his desires, or lay out his “demand function,” and command action be taken. Depending on how he judges the resulting action when compared with his updated expectations, the executive or commander can reward or punish his subordinates. This method of administrative control is often called after-the-fact control. However, when defense executives seek production from the open market, whether it be firms, universities, or non-profits, they must use market exchange mechanisms characterized by contracts. A contract essentially seeks voluntary agreement between two or more parties where the exact responsibilities of each party and methods for evaluation are detailed before action is taken. A similar method of administrative control is wielded by the controller using program budgets. Both contracts and program budgets use the method of before-the-fact control. Professor Thompson compared the two methods with respect to internal administration:

“[Before-the-fact] controls necessarily take the form of authoritative mandates, rules, or regulations that specify what the subject must do, may do, or must not do. The subjects of before-the-fact controls are held responsible for complying with these commands and the controller attempts to monitor and enforce compliance with them.

“After-the-fact controls are executed after the subject, either an organization or an individual, decides on and carries out a course of action and, therefore, after some of the consequences of the subject's decisions are known.”[1]

The congruence between the methods of contracting and program budgeting made the two natural bedfellows, the enabler being the unifunctional project office structure. The program budget demands that organizations find perfect alignment with program structure, which Mosher had showed impossible for any significant organization. These forces, coupled with an expressed desire for private production, led the Air Force to favor the systems project office and a single prime contractor. For the pre-existing Army and Navy organizations to be viable, there needed to exist an auditable accrual accounting system by program. Without such an accounting system, the policy maker could not effectively monitor execution to plan; nor could the policy maker forecast future plans. Though Congress mandated such an accrual accounting system in 1955, it was never accomplished.[2] Thus, the move toward unifunctional project offices can be seen as a means of outsourcing accounting compliance as well as production knowledge to industry. Control through the program budget would otherwise require multifunctional in-house organizations to perform such accounting themselves.

---

[1] Thompson, Fred. “Public Economics and Public Administration.”

http://www.willamette.edu/~fthompso/ECON&PA.html.

[2] Lander, Ezra. “Performance Budgeting and Accounting Policy in the Department of the Army.” The American University, Ph.D., 1961, pp. 242. Accrual accounting was first emphasized to the Congress in the Second Hoover Commission of 1955, but its need was understood much earlier. Despite repeated mandates, an auditable accrual accounting system remains years away in the Department of Defense.

Supercomputer to navigate regulations?

Here is Scott Chandler in a Lexington Institute article:

"The defense acquisition system is a construct of government, erected over decades and codified in statute that now exceeds 180,000 pages.  It is so complex that the Air Force commissioned a supercomputer to make sense of it."

I'll leave the 180,000 pages alone. The news article announcing the Air Force supercomputer to deal with the acquisition process should give one pause. Here is the Air Force justification:

"The Air Force pointed to a 2006 study by the Government Accountability Office, Congress’ top watchdog, which found that 'the challenge of operating in accordance with complex federal acquisition regulations discourages small and innovative businesses from partnering with the government in emerging markets.'

"The Air Force hopes the new system will be an advanced tool, making it easier for businesses to understand the requirements of a contract and to get any of their questions answered immediately by the computer system."

So to encourage small business and innovation in new markets... the Air Force wants a supercomputer to navigate the deluge of regulations? It's not clear what processes the supercomputer would execute, how all the rules will be simplified through number crunching, what information people could to "mine," and whether its biggest benefit accrue to existing prime contractors and lead to further consolidation.

A pristine farm of super computers

Here is a thought to chew on: the Air Force will "teach the system how to understand context so that it can answer questions accurately."

Technology readiness and the F-35

The Straus Military Reform Project has a new set of podcasts taking the name of their nice 2011 book, The Pentagon Labyrinth. One episode by Dan Grazier, "The F-35 continues to stumble," provides a good exposition of the F-35's current status. There are so many points, but here is one representative example:

"The cannon in the F-35A sits behind a small door on the side of the aircraft that opens quickly an instant before the cannon is fired—a characteristic intended to keep the aircraft stealthy. Test flights have shown that this door catches the air flowing across the surface of the aircraft, pulling the F-35’s nose off the aimpoint resulting in errors “that exceed accuracy specifications.”"

Read the whole article if you don't listen to the podcast. I won't go on here about well documented troubles of the F-35. But the gun issue reminded me of another matter.

I had recently been reading about what people said about the F-35 program back when it was going through milestone B in 2002. The hot topic of the day was "knowledge-based acquisition," where information on technological readiness drives the progression of program phases. The Technology Readiness Level (TRL) was developed as an industry best practice for measuring knowledge achieved. For example, programs should not enter full scale development (pass Milestone B) until critical technologies reach a TRL of at least 6, where system/subsystems have been modeled or prototyped in a realistic environment.

What's the point? Certainly no one was debating the technical readiness of the F-35 gun, and even if they did, it would be at the highest level of 9. But was anyone thinking about interactions between technologies, and how they would operate in their actual environment? Time and again it is not until a system or subsystem is put to the test that critical problems are exposed, and all our supposed knowledge implied by the TRL means little or nothing.

Cost Growth, a brief interpretation

Today’s quote is from page 273 of the Second Hoover Commission Hearings (1957):

“Mr. Brown. [The Department of Defense] seldom get anything for the price tag originally put on it…

“Mr. McNeil. That was certainly true some years ago… For about 4 years now, most estimates are on the beam. And it was always possible.”

It is hard to believe that between 1953 and 1957 defense cost estimates had been “on the beam,” particularly when Martin Peck and Frederic Scherer found substantial cost growth in the 1950s in their classic The Weapons Acquisition Process: An Economic Analysis. Maybe they were just better than the estimates developed during the Korean War crisis.

Cost estimation continues to be a major problem today. In fact, it is widely agreed that cost growth has come down relative to the 1950s period. I doubt any informed observer would say we have contained cost growth, or that we are doing better today than in the 1950s, despite the fact that the AT&L’s “Performance of the Defense Acquisition System” repeatedly trumpets cost containment.

It may not take a searching review to understand that cost growth need not have any relationship with program outcomes. As Robert Devons wrote, cost growth studies “showed that most of the so called cost growth was the direct result of low initial cost estimates.”

So, as Donald Srull said, “One could not after all, buy the new Cadillac automobile for $5,000. When it turned out to cost $25,000 it was not “cost growth”—it was an unrealistic, poor initial underestimate!” (Srull was the first chairman of the Cost Analysis Improvement Group, or CAIG.)

Our problem today is that in a decision maker’s mind, a higher cost estimate is a more “realistic” one. If a realistic cost estimate for a $25,000 Cadillac comes out to be $40,000, anything other than large cost underruns signal escalating backend prices. Unfortunately, “should cost” initiatives cannot deliver the required information. We only know a $25,000 Cadillac doesn’t cost $40,000 because there is a market for Cadillacs which is embedded in an even broader market framework. You cannot just add up the labor and materials required to produce technological innovation.

The problem more pernicious than cost growth is simply high costs (similar to the “cost disease” inflicting the commanding heights of education and healthcare). It might be stunning to some that the Bruce Harmon of the Institute for Defense Analysis found that the fighter aircraft industry experiences price escalation at nearly twice the rate seen in the economy at large. (In other words, for a constant quality fighter aircraft our “buying power” is falling by about 2%-3% per year—every couple decades we can only afford half as much with the same inflation-adjusted dollars.)

Harmon’s conclusion… the F-35 is an average performing program and it experienced “cost growth” because of its artificially low Milestone B cost estimate that did not assume fast enough price growth in line with experience! If the F-35 is not really a cost growth problem, then cost growth is not the problem we should be looking at.

Economics merely a branch of decision theory?

Bonus quote today is found on page 510 of Philosophy of Complex Systems, from an essay by John Foster called "Economic Systems" (2011):

"Over the past half century, economic theorists have stubbornly held on to their view that economics should be a branch of decision theory that involves optimizing choices along artificially smooth and conveniently specified production and utility functions, subject to constraints."

A lot was said in that sentence. Economists were "stubborn" because, despite the continual failure of their models to describe reality, they hold on to assumptions of "artificially smooth" and "conveniently specified" functions. They hold onto these methods because linearity is mathematically tractable (the same critique is sometimes levied on physics).

But clearly we live in a complex world dominated by numerous changing factors. For this reason, it is misleading to think of economics as "a branch of decision theory that involves optimizing choices" and instead one should think about how individuals interact and exchange.

As Armen Alchian concluded in 1954:

"Systems analyses are machines for generating implications of postulated initial information; they do not generate decisions...  Under uncertainty, the criterion of decisions is not simple maximizations; the essence of the decision process is to affect the scope of random factors so as to give a “good” probability distribution of outcomes. The insurance principle is to decisions what maximizations are to analytic implications."

His 1950 paper "Uncertainty, Evolution, and Economy Theory" argued that optimal decisions can only be regarded as such after-the-fact through a process of natural selection. Decision problems under uncertainty were solved, he later found, by maximizing the option space available. He agreed with Herbert Simon that one should optimize when one has the necessary information, but an evolutionary "satisficing" method probably more appropriate given the vagaries of our world.

Industrial Consolidation, Aircraft Series

      Today's quote is from "An Initial Look at Technology and Institutions on Defense Industry Consolidation."

     ".... nine prime contractors designed and flew 40 different fighter aircraft designs during the 1940s and 1950s (Lorell & Levaux, 1998)... However, as jet fighter technology matured the number of aircraft designs fell with less than a dozen U.S. fighter aircraft developed since 1960...  

     "A related observation that supports this statement is that there have been no new entrants into U.S. manned aircraft production since 1945 (Birkler, Bower, Drezner, Lee, Lorell, Smith, Trimble, &Younossi, 2003).”

Nassim Taleb on forecasting

The bonus quote today is from Nassim Taleb's The Black Swan, on page 163: 

"There are those people who produce forecasts uncritically. When asked why they forecast, they answer, “Well, that’s what we’re paid to do here.” My suggestion: get another job."

I hear a lot about predictive analytics and the like, but it's not clear that new fads made possible by big data and machine learning are more useful to complex organizations than earlier techniques. Most perniciously, attempts to supply the data and make use of the analysis injects so much rigidity that organizations are robbed of their vitality and initiative.

Edward Luttwak on Efficiency and War

Today's quote is from pages 136-38 of Edward Luttwak's The Pentagon and the Art of War: The Question of Military Reform, 1985.

“The divide between efficiency and effectiveness is at its deepest in combat. What are the most effective military operations? In very general terms, they are operations in which the enemy is not outfought by greater firepower, greater numbers, greater bravery, and greater sacrifice of lives, but instead surprised and then outmaneuvered and disrupted, and thus never given a chance to fully employ his fighting capacity in the first place.

“And what is the common denominator of such operations? Inefficiency 

“Consider surprise. How is it achieved? By deception – unless the enemy is merely apathetic or unobservant, and therefore outclassed to begin with. And how is deception achieved? By doing the unexpected. And what is the unexpected? Something other than the sensible, normal, and efficient."

Luttwak connects the strategic concept of inefficiency to weapon system acquisition:

"But conflict is not like civilian business and efficiency is the wrong goal to pursue: efficiency in making a radar or refueling a ship, of course; efficiency in making radars, or refueling ships, no, for efficient economies of scale in purchasing radars lead to a single mass-produced radar that will be more easily counter-measured, and efficient refueling leads to a few large fleet oilers that are more easily intercepted and destroyed by the enemy. (Each of our majestic aircraft-carrier task forces is now dangerously dependent on a single, very large, very efficient resupply ship.) Conflict is different.” 

The concepts do not just apply to conflict, but to all dynamic systems. Certainly the market economy, itself a dynamic system, fosters inefficiency due to competition and redundancy. Like handling the uncertainties of war, perhaps local inefficiencies produce overall effectiveness of the market system to adapt and progress.

Today's Quote

Is from pages 273-78 of James E. Hewes, Jr.'s "From Root to McNamara: Army Organization and Administration" http://www.history.army.mil/books/root/

"Technical service chiefs could and did transfer funds freely among their various activities, functions, and installations, but neither the Secretary nor the General Staff could legally transfer funds among the several technical services or other staff agencies without going to Congress for approval."

That was before passage of the 1949 amendment to the National Security Act. Title IV instituted the first "performance budget," or program budget that is the prototype of the Planning-Programming-Budgeting-Execution System (PPBES) in use today. It was the brainchild of Ferdinand Eberstadt and Wilfred J. McNeil, the latter becoming Assistant Secretary of Defense(Comptroller), for the next decade. 

Hewes goes on:

"With this one directive McNeil wiped out the independent budgets of the technical services dating back in some instances to the Revolution. The chiefs no longer would defend their budgets before Congress. Instead this would be the responsibility of the several General Staff divisions. Congressional restrictions on transferring funds among appropriations would hamstring the technical services rather than the General Staff."

Before 1949, the budget was controlled by independent heads of different Army organizations, providing competition and diversity -- particularly in the technical services which performed weapons procurement. Afterwards, it was to be organized centrally from Army staff under the authority of the Secretary of Defense and his Comptroller.

Perhaps the most pernicious part of it all, the performance budget controlled not just the flow of dollars, but the precise objectives those dollars were to accomplish. No longer could a project manager decide to take alternative approaches to new technologies on his own authority, and contract freely with industry.

Is there really $125B in Pentagon Waste?

Imagine you're the Deputy Secretary of Defense, and in walks a group of consultants who tell you that you can save $125 billion over five years if you act on their recommendations. They say you need a clear science and technology strategy, to establish rigorous management processes, and so forth. They say savings will start immediately. You flip through their PowerPoint brief stacked with charts and buzz words like optimize, modernize, and agile enterprise. You get weary when a slide shows, in pictographs, how you can re-program the efficiency savings to 50 brigades, 10 carrier strike groups, or 83 F-35 fighter wings over the same five year time frame. Slow down, you think. You don't want Congress getting a hold of the idea that there is $125 billion in waste can disappear overnight because you know there are no easy answers of the kind peddled by consultants.

That is, unless you're Deputy Secretary of Defense Bob Work. As Craig Whitlock and Bob Woodward reported in the Washington Post, Mr. Work did in fact get worried when he saw the chart depicting the savings. He got worried, however, that the DoD could easily save those billions and that Congress would want it back! The journalists wrote that the report by private consultants unambiguously laid out "a clear path" to $125 billion over five years. Former chairman of the Defense Business Board Robert L. Stein called the study's data "indisputable" because it came from the Pentagon itself. Because the DoD feared that Congress would "use the findings as an excuse to slash the defense budget," officials decided to suppress the "internal study that exposed $125 billion in administrative waste."

Frank Kendall, Undersecretary of Defense for Acquisition, Technology, and Logistics (AT&L), fired back that the consultants' study was "shallow" and failed to grasp basic obstacles to restructuring the public sector. The Washington Post journalists shrugged at the response and suggested that Pentagon officials were just trying to "wait things out" rather than tackle the hard issues at hand. It makes one wonder whether the journalists actually read the report. Perhaps a brief foray into the 125 billion-dollar report will give the reader a feel for the argument at hand.

The report, little more than a deck of PowerPoint slides, starts out by showing potential savings based on projected productivity gains. The consultants assumed 7% annual productivity gains to get to the $125 billion, whereas 3% annual gains only get you $60 billion in savings. Why 7% productivity? Because private sector industries "commonly show similar gains." Never mind the fact that economists generally agree that productivity has been flat for years. The industry benchmarks used in the report are not representative of defense support activities. Productivity in "office supplies, stationary, and gift store" is not representative of productivity in defense, even if they are DoD inputs. Moreover, there is no generally acceptable way of measuring productivity in the DoD, even for support functions. This is because the defense "marketplace" is only a loose facsimile of the kind you read about in economics text books.

Supposing that the DoD could achieve 7% productivity gains, how would the consultants recommend the DoD gets there? Productivity gains amounting to $46-89 billion can be achieved through "contract spend optimization" which requires more rigorous vendor negotiations, aggregating spending to gain economies of scale, increasing productivity, and eliminating "gold plating." This is basically all they had to say on regarding half of the total savings. The supposedly specific and actionable recommendations merely restate the objectives of existing job functions. For example, to anyone familiar with defense vendor negotiations, it is hard to believe that regulations on contracting officers could get any more rigorous. Lawyers are brought into the discussions at ever earlier stages in preparation for award protests. In short, the recommendations are empty.

In almost all cases, productivity gains were implied by consolidating information technology (IT) infrastructure and leveraging its output, big data. But even smaller firms find implementing IT solutions difficult, fraught with unforeseen challenges and cost overruns. Striving for bigger and better data is often a fool's errand, even for relatively simple operations.

An early example of the DoD implementing streamlined IT efficiencies came during the Korean War. The Munitions Board attempted to centralize the inventory data of machine tools using IBM punch cards to optimize allocations. The attributes of each machine were recorded and contractors could read the cards to locate their tools instead of doing on-site inspections. But machine tools are more complicated than the Munitions Board had assumed. The IBM cards completely failed to convey important nuances in machine tools leading to expensive errors and lost time. The initiative and countless other more like it sought to strip away complexity and replace it with the pristine order that consolidation imposes.

The out-sized share of the PowerPoint went to specific discussions on IT consolidations, whereas the consultants hand-waved issues of contracting and civilian fringe costs. This is concerning because the Task Group Chair, Mr. Phil Odeen, also has positions at Booz Allen Hamilton and Globant, both of which are IT services providers. According to the meeting minutes, he did most of the talking. Was the report a well-intended set of recommendations, a business pitch, or both?

The easiest way to see that the consultants were blowing smoke comes at the end. They write that "Technical solutions... will not, by themselves, achieve the savings." The technical solutions are a tool that must be supported by a culture change. For example, there needs to be "strong, consistent top leadership," "clear decision-making authority" and "defined accountability at all levels." Here, the consultants pay lip service to what really matters, and reference their earlier work on culture change. The only problem is that their conclusions are antithetical to all present discussions on defense reform!

The consultants want more centralized planning and more rigorous regulations. Congressional leaders, such as Rep. Mac Thornberry and Sen. John McCain, are calling for decentralization and fewer regulations. Legislation is being discussed would devolve some of the planning and programming decisions from AT&L down to the services. Further, AT&L itself may decentralize, separating its R&D functions from acquisition and logistics. The Pentagon is in the awkward position of being pulled in two directions.

Yet being pulled in two directions is par for the course in the Pentagon. For example, the landmark 1986 Goldwater-Nichols Act attempted to decentralize operations, but, as Rep. Thornberry said, it "unintentionally fueled a runaway, outmoded bureaucracy." The public should take little solace, however, that any prospective legislation would turn out differently. As scholar Fred Thompson found, decentralization requires giving operating managers the "maximum authority feasible, or, in the alternative, subjecting them to a minimum of constraints." But Congress doesn’t trust the lower echelons and wants to increase constraints! For example, the legislators want binding unit cost and schedule targets earlier in the acquisition process, at Milestone A, which is at initiation of Technology Development or prototyping. (Note that the troublesome F-35 went through a Milestone A equivalent in 1996. Certainly that is too early to smooth over programmatic risks with extensive before-the-fact controls.)

Ultimately, one must acknowledge that the fundamental principles of complex operations are decentralization, redundancy, and self-organization. While policymakers admit the inherent complexity in defense and claim they desire to decentralize operations, they shy away from actual implementation because of its superficial illogicalities. Hence they attempt to centralize, eliminate duplication, and control operations through extensive regulations. While in many ways it mirrors the de facto way of doing business in the 1940s and 1950s, a return to direct budget appropriations to decentralized operating units who are controlled using after-the-fact evaluations would go a long way to effecting real defense reform.

But such decentralization means a loss of coherence at the top. It means you should be not expect a reasonable answer from seemingly simple questions. Sen. McCain, for example, fumed that "The DoD cannot tell us how many civilian contractors they employ, no one knows how many people work for the Department of Defense." While that line was trumpeted around as a clear indication that the DoD didn't have its management in order, one has to sympathize with the poor analyst in some cubicle in the Pentagon who knows that the data are flat out not there, nor should they be. Just for starters, are we talking about direct charging workers from prime contractors, or down to the second tier, or third, or fourth, and so forth? Do we include indirect workers? Or which types of indirect? Are we talking physical headcounts or full-time equivalents based on hours charged? The analyst would think, “For there to be any chance of having a clean answer at my fingertips, it would require far more comprehensive cost controls.” The Senator incentivized defense officials to cover themselves with more centralization and new regulations at the same time he cried that the DoD is too centralized and bloated with regulations.

The answer turned out to be about 740 thousand. But the better question is whether that matters. Is there any basis for judging 740 thousand contractors as too high or too low, even if the number were not arbitrary in some sense? The Sen. McCain’s statement makes clear that he views the defense enterprise as a vast administrative problem, where decentralized operations can be neatly aggregated to inform policy. In this way, the Senator champions the utopian view of consolidation and big data at the same time he champions decentralization and competition. At least the consultants who would save $125 billion on baseless assumptions were consistent, if shallow, in their views.