Use Data to Improve Medical Care at Sea
l (CVA-59) began air operations against North Vietnam. In the days before she arrived on Yankee Station, several accidents had occurred on her flight deck, including the unintended ignition of a large flare and the maiming of a Sailor run over by a plane. On 28 July another Sailor was lost overboard; a day later, the accidental firing of a Zuni rocket ignited the catastrophe that killed 134 and nearly destroyed the ship.1 Before the war ended, other major aviation accidents and fires would occur on board the USS Enterprise (CVN-65), Hancock (CV-19), and Oriskany (CV-34).2Naval aviation has come a long way since then. Thanks to a revolution in safety and quality management, our flight decks are now the world’s showcase of safe operations under hazardous conditions. During a cruise on the USS Iwo Jima (LHD-7), where I served as senior medical officer in 2003–5, more than 11,000 helicopter takeoffs and landings were conducted without serious mishap, and not a single Sailor was lost. Medical care is in need of a similar revolution.
Borrow from Naval Aviation
In 2000 and 2001, two landmark books by the Institute of Medicine (IOM) of the National Academy of Sciences detailed the horrific safety and quality record of the current U.S. health-care system.3 The IOM cited studies showing that 36 percent of hospital patients had suffered harmful medical accidents, and that 27 percent of hospital deaths might have been prevented.4 The conclusion was that medical errors in hospitals caused up to 98,000 deaths per year in the United States, making this the eighth leading cause of death.5 These issues are just as real for medical departments at sea as for any other facility.
Fortunately, fixes for these problems are detailed in the IOM books and a large body of supporting literature. What is striking is that many of these advances, as the IOM acknowledges, originated in naval aviation.6 Among them are simplification and standardization of processes, the use of standard operating procedures, automation of sensitive activities, and multiple layers of checks. All of these are exemplified by the counting of wheel chains before launching an aircraft. As a medical analogy of the latter, advanced hospitals now require the surgeon, nurse, and technician to certify independently which extremity is being prepared for surgery, wrong-limb operation having become an all-too-common event.
Another such application is data-driven quality analysis, which detects and eliminates unwanted variation from performance standards. These concepts are familiar to most in aviation, but only now are they being applied to medicine. On the Iwo Jima we pioneered such a system, based on methods developed by the Institute for Healthcare Improvement (www.ihi.org), a leader in quality innovation. The idea is to choose metrics that reflect medical safety and quality, and then measure and graph these frequently over time. The result is a “fever chart” (a graph showing temperature over time) of various aspects of medical care. This process also provides a built-in system for quality improvement. If a proposed change in methods is genuinely an improvement, it will drive the fever chart in the right direction.
Criteria for Medical Care
The Institute of Medicine has defined six dimensions of medical quality. Care should be safe, efficient, effective, timely, equitable, and patient-centered. These “IOM criteria” have become touchstones for the medical-quality movement. On the Iwo Jima, our method was to track first one, then multiple metrics for each category.
• Safety: Medication errors are a common cause of harm to patients. In Iwo Medical, once we decided to track monthly adverse drug events (including prescription errors), a surprisingly large number was found. System problems were responsible, not substandard personnel. Improvement was shown after corrective changes were introduced, including corpsman education, prescription guidelines, and review of all scripts by the pharmacy technician. Other metrics that could be tracked include errors in surgery or such “fault lines” as transitions of care from one health facility to another.
• Efficiency: We thought we were more efficient than proved to be the case. Total “cycle time” (the time an average patient spent in Medical) was nearly two and a half hours. Moreover, most of this was dead time spent waiting for a provider or technician. A series of improvements such as phone consultations; same-day appointments; and use of care teams combining doctors, nurses, and corpsmen cut dead time by 50 percent, releasing Sailors back to their duties. Other metrics could track the use of physician extenders (e.g., corpsmen) or management of patient problems by e-mail.
• Effectiveness: Effective care is based on objective evidence rather than the hunch of the provider. Providers may use a database summarizing the results of clinical trials, but these systems are not always user-friendly. The next step is Palm Pilots for all providers, with access to Web sites with the latest treatment recommendations.
• Accessibility: Because our open-access system already guaranteed same-day admission to Medical, we did not need to study this parameter. For facilities using conventional appointing systems, a good metric to track is the number of days of waiting time.
• Equity: We studied this problem by tracking differences between categories of personnel in attacks of diarrhea. We found that discrepancies could be traced directly to overcrowding in the enlisted heads.
• Patient-centeredness: Monthly satisfaction surveys are good ways to keep the patient at center stage. Our charts indicated that most patients appreciated the decreased waits in Medical, despite the elimination of sick bay as a place to take a rest.
Break Down the Data
Ideally, several metrics will be tracked for each area of quality. One may choose whatever metrics seem useful on a given ship. When the usefulness of one is exhausted, another can be chosen. Such a matrix gives a continuous “report card” on medical-department performance. More important, it diagrams the effect of changes in the process of care. This allows continuous and progressive quality improvement.
The skills required for such a project are simple arithmetic and graphics. They are well within the powers of the average ship’s doctor or independent duty corpsman. The concepts in this article are components of the Patient-Centered Medical Home model of care, endorsed in 2009 for all military health system facilities. Accordingly, they are consistent with the strategic direction of Navy medicine as a whole.
There is no reason why such worthwhile systems should not become standard operating procedure in all U.S. warships. In terms of safety and quality, our flight decks are 40 years ahead of our medical departments. By addressing the points in this article, commanders can make that gap disappear.
1. G. A. Freeman, Sailors to the End (New York: William Morrow, 2002), pp. 62-102.
2. R. R. Burgess, ed., The Naval Aviator’s Guide, 5th ed. (Annapolis: U.S. Naval Institute Press, 1996), p. 39.
3. L. T. Kohn, J. M. Corrigan, and M. S. Donaldson, To Err is Human: Building a Safer Health System (Washington, DC: National Academy Press, 2000). Committee on Health Care in America, Institute of Medicine, Crossing the Quality Chasm: A New Health System for the 21st Century (Washington, DC: National Academy Press, 2001).
4. Kohn, Corrigan, and Donaldson, To Err is Human, pp. 31, 33.
5. Ibid., p. 1.
6. Ibid., pp. 160-62.
We Need a User-Centric Navy
In a December 2009 Signal magazine interview, Chief of Naval Operations Admiral Gary Roughead stated: “The speed of communications and the rapidly changing picture of the battlespace are the main drivers for the Navy’s new info-centric approach . . . The Navy must be more focused on how information is derived, developed and moved to the right customer at the right time, in the right way.” Readers might reasonably expect the admiral’s remarks to then address the critical problem of improving the quality and timing of information for Navy warfighters. Instead, he drills down on a technical issue: “We in the Navy can talk about a bandwidth problem, but I don’t believe we cannot say it is not an antenna problem.”
Once again, the significant issue of meeting warfighters’ needs was overlooked. Lack of any focus on the user is one glaringly obvious reason why service members continue to be over-tasked when they try to use the systems provided, resulting in numerous challenges and frustrations as they simply try to do their jobs.
Who Is Using the Technology?
For many years we have focused on technology in system development programs, typically with almost total disregard for the user. To be successful, the info-centric warfare approach that Admiral Roughead espouses must include a shift in priority from developing technology to empowering the warfighters who use it.
The typical systems-engineering-driven approach for net-centric system design, such as that of the Consolidated Afloat Networks and Enterprise Services, focuses on hardware, software, networking, and security. The method rarely includes initially performing a comprehensive analysis of users’ capabilities and limitations, or completing a rigorous quantitative set of analyses on, for example, tasks and workload/workflow to identify and understand the users and their real needs, before starting on the design effort.
Systems-engineering teams may believe this is being accomplished when they ask groups of users what they want. However, that course of action rarely collects more than divergent qualitative opinions. The quantitative approach to the data collection needed, and one that should form the baseline for rational decision making on system-design issues, involves analyzing the mission, tasks, critical tasks, workload, workflow, and training needs. Typically these aspects are glossed over because systems-engineering managers believe such work takes time and money away from the “real” job of designing the hardware and software.
The resulting designs are data-intensive systems that overwhelm the abilities of our warfighters to manage them. It is easy to see why.
Without a good understanding of users’ capabilities and limitations, specifically what the users need to do, how they do it, and exactly what they need (functionality), systems engineers tend to over-specify and over-design to include every possible piece of functionality that can be included in the product, given the hardware- and software-driven constraints such as size, weight, electrical power, cooling, and so on, and portray this plethora of functionality as a benefit. For example, digital cameras are designed with 300+ functions, while the typical user actually needs about 5.
The Data Must Be Accessible
The real effect of info-centric system development is that floods of data—versus useful information—drown our warfighters and reduce their effectiveness. We have developed the technical capability to move vast quantities of information quickly, store it, secure it, and have it readily available for the warfighter. But for info-centric warfare to be successful, we must now focus on the human factors. We must more fully understand users’ abilities and limitations as related to this vast quantity of info-centric data, and identify their specific needs. Info-centric-system design should start with answering some key questions:
• What are the users like? What can the warfighters do and not do?
• What information do the warfighters need to do the job (and what do they not need)?
• How should the information best be delivered to the warfighter?
• How should it be formatted?
• Collectively, how should the data be presented to be easily understandable, actionable, and effective?
The starting point for the solution lies in the Navy’s acquisition community. The Interim Defense Acquisition Handbook states: “For all programs regardless of ACAT [acquisition category], the PM [program manager] shall initiate a comprehensive strategy for human-system integration (HSI) early in the acquisition process.” The instructions continue:
The PM shall employ human factors engineering (HFE) during systems engineering (to include function allocation) to provide for effective human-machine interfaces. Where practicable and cost effective, design efforts shall seek to reduce manpower and training requirements. Design efforts shall minimize or eliminate system characteristics that require excessive cognitive, physical, or sensory skills; require extensive training or workload-intensive tasks; result in mission-critical errors; or produce safety or health hazards [emphasis added].
Unfortunately, recent experience indicates that Navy acquisition professionals are generally ignoring these directives when drafting requests for proposals (RFPs) for acquisition of systems. Many examples support this assertion.
Write RFPs with People in Mind
Recently, a Navy acquisition command issued an RFP for engineering-support services. In contrast to the guidance provided by the handbook referenced here, this RFP barely mentioned human-factors engineering and did not address human-system integration at all. In this instance, a potential bidder provided to the contracting officer information on HFE and HSI guidance from the handbook, along with recommendations on how the RFP’s language could be modified to comply with it.
The contracting officer acknowledged the handbook guidance—and specifically chose not to modify the language to require the winning prime contractor to address HFE and HSI issues. When it is not in the contract, there is no requirement for the contractor to do it. Thus, it will not be done.
What is the impact? The F/A-18 community indicates that pilots and naval flight officers are task-oversaturated in trying to accomplish their missions. The F/A-18E/F and F-18G models will likely exacerbate this situation, as there appears to be no user-centered design emphasis for the cockpit. The likely result will be that aircrews will continue to be extremely overloaded and overtasked in trying to execute assigned missions.
As noted previously, the Interim Defense Acquisition Handbook specifically states that designs must avoid the need for unusually high skill levels to use the systems. Even though this directive should be followed closely, typically it is not. The prime contractors and system integrators build systems that contractually comply with the functionality requirements, but do not address problems of “excessive cognitive and sensory workload.” Thus they are likely to result in systems that require “excessive cognitive, or sensory skills, extensive training, workload intensive tasks,” and may result in “mission critical errors, and produce safety or health hazards.”
Human Engineering Is a Real Thing
Most RFPs written for acquisition of systems require little or no HFE or HSI work. If these aspects are addressed at all, the requirement is typically to develop a human-engineering plan. The prime contractor develops this plan to comply with the contract data requirements list. With that box checked off, the human-systems engineering plan is then put on the (electronic) shelf, and that is the last anyone ever thinks about it.
At a minimum, the RFP should include a requirement that the prime contractor actually execute what is in the human-engineering plan. Navy program managers and their staffs must be much more rigorous in ensuring that this occurs. They must also apply user-centric approaches to system design by completing the HFE/HSI analyses (mission, task, workload, workflow, etc.). This will turn around the trend of developing systems that meet the specifications—and are virtually unusable from the point of view of the warfighter.
The need for an info-centric Navy might be better stated as a need for a user-centric Navy. A refocusing of our approach to warfighters as the central part of the issue is they key to success. This shift must be initiated from the top, where leadership demands the user-centric approach to system design, rather than continuing to focus on the development of hardware, software, network systems, and security
To use an analogy from ancient sailing days regarding the adequacy of the propulsion system, the perspectives of a crew member beating the drum and one pulling the oar are different. Warfighters are less effective than they could or should be because we are not focused properly on the users.
Every system development should begin by conducting an analysis of user-centered requirements and committing very early in the engineering process to conduct the needed mission, task, workload and workflow, and training-needs analyses so that we have a sound foundation for developing the functional and system requirements. The users must become the central focus for system development. User interface must become a central part of the system-design process, rather than looking at the people as afterthoughts.
Captain Chamberlain works on human-system interaction issues in a variety of programs. He served as a naval flight officer in E-2Cs and as CO for the CINCEUR Reserve Unit, the AADC Pacific Unit, and the Battle Group 3 Staff Reserve Support Unit.
Re-Train the Crew before You Leave
Some say a commanding officer’s legacy is the ship’s performance six months after the change of command. This approach misses the point, by underestimating both the impact of a new commanding officer and the crew’s ability to improve (or atrophy) quickly. With these realities in mind, you, as the outgoing commanding officer, should aspire to bolster your relief’s success.
Take a Hard Look
Starting approximately one year before the change of command, you should do several things above and beyond the various requirements and checklists, including a good assessment. Scrub personnel issues—training, qualifications, watchbills, and rotation plans. Make difficult decisions to resolve personnel problems. Finally, examine the way you exercise command now, versus at the beginning of your tour.
All commands, even the top-performing ones, have strengths and weaknesses. Conduct an intellectually honest assessment to identify them. The goal is not to achieve perfection before the change of command. The pursuit of such an unobtainable standard will demoralize the crew. Instead, weaknesses indicate relative degrees of risk that require mitigation strategies including training and qualification plans, focused supervision, formal schools, and watchbill management.
Also, don’t underestimate the value of good old-fashioned practice. The point is to do the assessment and take action in time to make a positive difference.
A unique feature of this particular assessment should be an evaluation of where you, as CO, have had to insert yourself and why. Focused training and practice can reduce the number of times CO intervention is required, but the need will never be eliminated. It is important to let your relief know where you look for problems and why. That person can then benefit from your experience and be on track to succeed.
Management Nuts and Bolts
The lion’s share of preparation for change of command following an assessment should involve personnel management. In particular, look closely at qualifications and proficiency, rotation planning, and resolution-of-performance problems.
You will serve the new CO well if you execute a qualification plan that includes contingencies and backup. This promotes better performance, quality of life, and job satisfaction, among other things. With turnover in mind, the first step is to determine who should qualify for key watch stations that require the CO’s blessing.
Then set timelines and goals that support qualification, with time to gain proficiency before turnover. This strategy reduces the new CO’s risk of having to sign a watchbill before getting to know the crew.
The realities of personnel rotation will soon stress even the best-performing crews and the most well-executed qualification programs. Identify key players in critical watch stations and in important collateral duties. Plan for their turnovers. The reliefs for these personnel should be identified early so they can be trained properly.
Take Action before Turnover
It is useful to put the new personnel in place before the change of command, with the outgoing personnel still on board. This way the experienced CO trains the inexperienced personnel, who are supported by their predecessors. Long-term success is more likely.
Persistent performance problems must also be resolved before turnover. Again, early action is the key to success. Deliver the necessary counseling, Letters of Instruction, and so on, with enough time to evaluate progress and make course corrections before you detach.
Leaving the problems for the new CO, hoping that a perceived fresh start will do the trick, rarely works as intended. Allowing personnel the opportunity to improve under your guidance gives them a better chance to make a good first impression with their new boss. Taking the time to make the hard decisions on your watch gives the next CO a better team and is fairer to the individual Sailors.
Address the Hard Questions
The way you deal with people at the end of your command tour probably differs from the way you handled things at the beginning. In particular, you probably ask fewer questions about routine matters as your tour proceeds. After all, you asked all the hard questions early on and trained the crew to your standard. These actions reduced risk on your part, allowing you to divert your energy to concerns carrying greater risk.
A new CO will ask more questions because that person is less knowledgeable about the crew’s capabilities. This change can catch a crew by surprise, or even be misinterpreted as distrust. Pave the way for the new CO by asking the questions you did when you were new. Tell the crew what you are doing and why. This way they will be better prepared to interact with your relief.
Even though these recommended actions reflect the way business should routinely be conducted, many COs change the way they do things a year or so into the job. Command is no longer new, and COs get comfortable. Refocusing attention on the basics in preparation for a change of command can help position the crew and new CO for success. But after that, it is up to the new CO to make good decisions and take advantage of these opportunities.