 |
| Kaveh Safavi, M.D., J.D. |
Traditional measures of return on investment for health care IT expenditures count transactional efficiency gains and elimination of unneeded resource use. An important but harder to measure return comes from avoiding costs due to better clinical outcomes.
Given the costs of large-scale health care IT implementations, a meaningful ROI often requires demonstrating lower cost through better care--not just more efficient care.
A recent analysis of national hospital clinical performance has shown that in-hospital death rates have improved steadily in the last six years. However, the same cannot be said for measures of patient safety.
A study by John Griffith and his colleagues at the University of Michigan published in the November 2006 Journal of Healthcare Management found that between 1999 and 2003, the risk-adjusted, in-hospital death rates for Medicare beneficiaries improved by 9 percent, but the risk-adjusted patient safety index, a composite measure of 15 patient safety indicators developed by the Agency for Healthcare Research and Quality, worsened by 10 percent during the same period.
Perhaps more important, the analysis found that 29 percent of all hospitals showed significant improvement in in-hospital mortality, and 4 percent showed significant deterioration. However, only 6 percent of hospitals showed an increase in safety, while 19 percent showed a significant decline.
All of this has occurred in the face of increased adoption of clinical information technology designed to hardwire best clinical practices and improve outcomes.
What does this mean for the widely accepted hypothesis that adoption of clinical practices and technology are required to improve the quality of care? Does this finding disprove the hypothesis or support a change in approach toward demonstrating a return on investment for health care IT spending?
Understanding the Contradiction
Is hospital patient safety getting worse, or does it just look that way?
The apparent improvement in clinical mortality at the same time in-hospital adverse occurrences deteriorate can be used to prove many different points. One interpretation might be that too much attention has been focused on preventing in-hospital deaths at the expense of things that lead to complications but not death. This is hard to accept given that hospitals as a whole have increased their organizational focus on clinical quality.
Three other factors may be at play here. First, there may be an increased level of attentiveness to coding patient co-morbidities, driven by a desire to more accurately apply patient-level risk-adjustment models used to judge clinical performance. This, in turn, may result in better documentation of conduct that worsens measures of patient safety.
Another factor may be that hospitals are using electronic technology to more efficiently collect adverse events as part of improving their care process. Moving from paper to technology-enabled data management programs means fewer lost adverse care reports and better documentation.
However, perhaps the most important explanation is the effect of a hospital’s development of a culture of safety. Most clinical performance improvement initiatives include implementation of clinical information systems and are accompanied by a change in the way hospital clinical staff function. Given the increased focus on patient safety in the last five years, it is reasonable to believe that hospitals that undertake initiatives to reduce deaths are also embarking on efforts to improve safety. One of the most common observations in hospitals that move from a traditional medical hierarchy to one that is transparent and safe is a significant increase in the number of medical errors reported before improvement occurs.
The impact of the spike can be dramatic. Missouri Baptist Medical Center reported this phenomenon in the June 2005 issue of Quality and Safety in Health Care. The hospital found a fourfold increase in the reporting of medical errors and the occurrence of adverse events during the first two years of its safety improvement effort.
However, what this pattern really told Missouri Baptist was that the baseline rate was not the observed rate of events at the beginning of the program. It was actually the rate of events at the peak of reporting. Since this initial analysis, the hospital has continued monitoring the frequency of adverse events. John Krettek, M.D., the hospital’s chief medical officer, observes that in six years, the institution has reduced adverse drug events sixteenfold.
If Missouri Baptist had accepted its reported adverse event rate pre-program, the facility would have concluded that all of the efforts of its people, processes and technology had increased its number of adverse events, rather than decreased it. Adverse events that previously were undetected are now identified and subjected to intensive improvement efforts. As a result, the global adverse event rate has been reduced by nearly sevenfold.
Implications for ROI Measures
Hospital leaders who embark on large-scale investments in programs and technology can learn several lessons from these observations:
- An institution will look worse before it looks better. This is part of the open and honest environment that is needed for a real clinical transformation, with or without technology.
- Baseline measures should be chosen wisely. Pre-implementation rates may not be valid. Rather, it may be peak rates reported post-implementation that indicate the true starting line.
- The ROI horizon should be selected carefully--a timeline of several years, not next year.
- If a spike in reported adverse events is not seen, the hospital may not have completely adopted a culture of safety, which means that it will not get the most out of its technology investments.
For public policy leaders, the implications of this finding may be that improvements in measures of patient safety should be expected to follow improvements observed in mortality. The apparent contradiction revealed by Griffith’s analysis might simply be a lag. After all, mortality is a hard endpoint and not subject to documentation judgment the way co-morbidities and adverse events are.
Understanding this measurement bias makes it possible to understand that the quest for ultrasafe hospitals will mean we will appear to get worse before we get better.
Kaveh Safavi, M.D., J.D., is chief medical officer, Center for Healthcare Improvement, Thomson Healthcare, Evanston, Ill.
GIVE US YOUR COMMENTS!
HHNMostWired welcomes your comment on this article. E-mail your comments to hhn@healthforum.com, fax them to Most Wired Magazine Editor at (312) 422-4500, or mail them to Editor, Most Wired Magazine, Health Forum, One North Franklin, Chicago, IL 60606.
If you would like a FREE Subscription to Most Wired OnLine, please click here to register.
This article first appeared on April 11, 2007 in HHN's Magazine online site.
To respond to this article, please click here.
