This html article is produced from an uncorrected text file through optical character recognition. Prior to 1940 articles all text has been corrected, but from 1940 to the present most still remain uncorrected. Artifacts of the scans are misspellings, out-of-context footnotes and sidebars, and other inconsistencies. Adjacent to each text file is a PDF of the article, which accurately and fully conveys the content as it appeared in the issue. The uncorrected text files have been included to enhance the searchability of our content, on our site and in search engines, for our membership, the research community and media organizations. We are working now to provide clean text files for the entire collection.
The December 1994 issue of Proceedings contains an article by Lieutenant Colonel Mark Cancian entitled “Are You Telling the Truth?” In it he addresses the question, “How optimistic can an estimate be before it is, in effect, a lie?” Program managers are the primary target. Having been the recipient of one of Senator Proxmire’s Golden Fleece Awards— and having wrestled with the fallout from this problem for four years as program manager of the Deep Submergence Systems Program (DSSP) (1966-1971)—I believe some comments may be of interest to those who will be facing this problem, which will not go away.
Colonel Cancian states that “using the low estimate constituted a lie,” but in the course of a truly new development, there are many estimates—and they are used for different purposes. Whether any given estimate is a “lie” depends on how, when, and by whom it is used.
In the case of the nuclear-powered research submarine AT?-/, the original “estimate” was $30 million. This number was given to Congress by Admiral Hyman Rickover based only on the availability of that sum in the Polaris Management Fund and hence his request for approval— with no need to go back to Congress for more appropriations.
Congress approved this project without the usual Navy program, and the Chief of Naval Operations found out about it in the press. It fell to my lot to go back and seek additional money; the final bill (identified by audit) was S96 million. The original estimate was simply a guess based on the availability of money and Admiral Rickover’s desire to build the submarine.
In the case of the deep-submergence rescue vehicles (DSRVs), the trail of estimates was longer—beginning with the suggestion by the Deep Submergence Review Group that a submarine- rescue vehicle could be developed from Alvin (the then most successful deep-diving research submersible) and using the cost of that vehicle as a basis for discussion. They estimated $2-$3 million for a new vehicle—detailed characteristics to be developed. This was a “starting” or “discussion” number used by the group looking at needs in light of recent submarine disasters. It did not come from an engineering estimate. It was, however, used by Senator Proxmire via the Golden Fleece Award to criticize Department of Defense management.
By the time the program was approved and preliminary engineering studies were complete, the estimated cost of the vehicle had grown to about $12 million—a reasonable starting point comparable to a commercial development program approval by a chief executive officer or board of directors. The final cost per vehicle was closer to $36 million (a difficult number to extract because the program included many other facets, but a reasonable one for this comparison).
More or less concurrent with the Deep Submergence Systems Program were five civilian programs that were developing their own deep submersibles. The deep ocean was being touted as the next frontier, and the race to capitalize on this led to real competition in the development of the necessary technology. Each concern was developing a vehicle independently, to its own plans and specifications, with no Department of Defense input aside from the normal exchange of technical information available to all.
Because of the intense scrutiny and criticism of DoD management, I explored the problem with all of my civilian counterparts—the managers of each company-funded civilian program. Each was facing the same problems in this totally new and undeveloped field.
The question I posed was, What was the cost growth from the time you went to your board of directors and requested approval to build your submersible at X dollars until vehicle completion of trials? The average was a little more than three times the original estimate. This was consistent with our experience on both NR-I and the deep-submergence rescue vehicle. It also was consistent with a similar review of ten DoD weapon development programs, which experienced a growth factor of 3.1—somewhat lightly referred to as “Pi” by some of my contemporaries.
I would note that today, according to press reports, this cost growth is reasonably comparable to that experienced by the Denver airport, the Senate office building, the Miami airport expansion, the Super Collider, and other civil works that presumably do not suffer the problems inherent in the management
of Department of Defense projects. These major civil projects are not reaching into the unknown—they are large but use proven technology—and they all have had the problem of what to believe in the development phases.
In the Deep Submergence Program (as in the space program and many emerging Defense development programs), we needed to develop and prove hardware that had never been built. Today, we have 30 years of developmental experience in these arenas to aid in estimating, but every step forward involves unknowns. Engineers may have high confidence, based on this experience, that new problems can be solved, but the dollar costs of solving these problems with little hard-data background remain very uncertain. The problem for the program managers is how to handle this uncertainty, once they have been assigned to get the job done.
Experience would seem to indicate that a factor of three would be a prudent allowance, but what executive or board would approve a plan in which the proponent starts with a 66% cost uncertainty? Thus, we find intensive pressure to go with hard, tight estimates—“best estimates” if you will—but with the expectation that as the feasibility becomes apparent, additional funds can be justified on a case-by-case basis. This essentially is why such programs usually are handled under the research-and-development umbrella, where developmental costs can be reviewed on a case-by-case basis. The entire Deep Submergence Program was so handled, with the result that there were many estimates and reviews—none of which were advanced with intent to deceive. The question nearly always was, What is the least cost to complete? The answer usually was, “Based on everything we’ve done to date, and if all goes as planned, it will be. . . .”
This problem underlies many of the widely publicized critiques, and being among the leaders in trying to develop totally new systems, the Defense Department is particularly vulnerable to such uncertainties. It is of great importance that such re
Commentary
Perhaps the factor of three should be recognized at the start—a sort of "Murphy's Law" for new programs—to be applied to the budget whenever the task to be undertaken has no history of returned actual costs available for reliable cost estimating.
search-and-development programs be evaluated with recognition of the uncertainties inherent in exploring the unknown. Such estimates are not lies; they are the best compromise possible, giving a reasonable balance between needs, knowledge, and availability of funds.
There is a vast difference between the numbers kicked around the table in early feasibility discussions and the budget number approved at the official start of any program. The latter ought to be based on the best available engineering analysis—it is the number to which the program manager must work—but it inevitably will include areas of uncertainty, and the extent of these should be recognized. Such uncertainty hardly constitutes lying if it is stated up front. We have had enough experience with this problem to alert all of the participants to reality.
The search for an acceptable set of numbers for a new development, a new production contract, or a contract extension involves at least three parties: the party in need (in our case, the Chief of Naval Operations), the party expected to do the work (the program manager or contractor), and the provider of funds (Congress). Negotiations between and among these parties are not unlike Cancian’s bazaar, and, like his example, none of our participants should feel deceived in the bargaining process.
Perhaps the factor of three should be recognized at the start—a sort of “Murphy’s Law” for new programs—to be applied to the budget whenever the task to be undertaken has no history of returned actual costs available for reliable cost estimating. This budget reservation in the out years would not relieve the program manager of cost-control responsibilities, but it would clear the air for a realistic discussion of expected problems. If the groundwork for such an approach could be laid in Congress, it would relieve the program managers of the nearly
constant need to fight for more funds and might even make it possible to release reserved funds when the job can be done for less—a truly novel notion. This was the broad intent of the Polaris Management Fund; it allowed the manager to concentrate on getting the job done. It also allowed the diversion of uncommitted funds to start the NR-I's construction.
Colonel Cancian condemns the service as “being less than honest, indeed deceptive. Its estimates are systematically wrong in a way that benefits the service’s self-interest—parochial and self-serving.” I agree with his conclusion that “Truth is a matter of perspective,” but I would add that Truth, like Beauty, is in the eye of the beholder; and there are at least three players in this high-stakes game in which the program managers are the servants, not the masters.
Captain Nicholson was program manager of the Deep Submergence Systems Program before retiring from active duty in 1971. He spent the next ten years with NOAA as associate director of National Oceans Survey until his retirement from federal service in 1981.
The U.S. Naval Institute Announces the 34th Annual Naval & Maritime
PHOTO CONTEST
$500
$350
$250
$100 each
The U.S. Naval Institute is proud to announce the 34th Annual Naval & Maritime Photo Contest.
The contest is open to both amateur and professional photographers. The winning photographs will be published in a 1996 issue of Proceedings, the monthly magazine of the Naval Institute. Cash prizes will be awarded as follows:
1st Prize 2nd Prize 3rd Prize
Honorable Mention (15)
Entry Rules:
1. Each photograph must pertain to a naval or maritime subject. (The photo is not limited to the calendar year of the contest.)
2. Limit: 5 entries per person.
3. Entries must be either black-and-white prints, color prints, or color transparencies.
4. Minimum print size is 5” x T.
5. Minimum transparency size is 35mm. (No glass- mounted transparencies, please.)
6. Full captions and the photographer’s name, address, and social security number must be printed or typed on a separate sheet of paper and attached to the back of each print or printed on the transparency mount. (Do not write directly on the back of a print. No staples, please.)
7. Entries may not have been previously published, and winners may not be published prior to publication in Proceedings. Prior publication could result in the relinquishment of the prize awarded.
8. Entries must be postmarked by 31 December 1995. Only photographs accompanied by self- addressed, stamped envelopes will be returned. Photographs not awarded prizes may possibly be purchased by the Naval Institute.
Deadline: 31 December 1995
Write for details or mail entries to:
Naval & Maritime Photo Contest-U.S. Naval Institute-118 Maryland Ave.- Annapolis, MD 21402-5035-(410) 268-6110