Episode 2, Identifying the Problem & Benchmarking. We've talked some about identifying a problem in the previous module. But we'll briefly revisit the basic concepts now in order to link together all aspects of the quality improvement process. As we noted the choice of an improvement project is arguably the most important step. Climbing a ladder that's against the wrong wall will seldom be of any value. There are many ways in which problems or opportunities for improvement can come to our attention in healthcare. Sources might include a review of results on performance measurements done internally, or required by health plans or accrediting bodies. Concerns identified by patients through complaints or other means. An error that caused harm to a patient. Accreditation reports, malpractice actions, or simply someone's observation of what appears to be a major problem. Whatever the source of information for identifying a problem, there are important factors to consider. First, the problem should be seen as important by a fairly broad set of stakeholders starting, in most cases, with our own patients. While this seems obvious, there are many instances in which a great deal of time, effort and resources have been expended on improvement efforts, then in the end seemed unimportant to the practice, to the patients, or to those other stakeholders. Secondly, if the problem definition did not include data to begin with, some data will need to be defined and gathered to clarify the situation and establish a baseline of performance. Selecting or addressing the aspect of the issue that shows both wide variation in performance between units, or over time, as well as having a relatively low average performance is often helpful. There should also be strong evidence, or at least good clinical logic, that higher and lower performance is related to some process that can be controlled or at least influenced within the health care domain itself. There are many regrettable experiences where improvement projects have floundered by focusing in on an area with little variation, or where performance was already very high prior to any intervention. Or finally where there was really no intervention within the health care setting that really had a major impact on the results of the process. If formal improvement work is relatively new to you or to your organization, it's useful to choose a relatively simple intervention that has been successfully completed In similar settings. It's also helpful to choose a project in which the intervention is fairly simple, straightforward, and where the results can be seen fairly quickly. The improvement scene is littered with failed projects that were more complex than they had to be, or that required many months or even years to see clear results, causing a loss of interest and enthusiasm, and eventually abandonment of the effort. A good resource for finding successfully completed projects is the AHRQ Innovations Exchange website, or reports on collaborative projects from the Institute for Health Care Improvement. Let's assume that you have access, the results from measurement being done in your own practice, and you have identified an important problem. You feel that there are a number of processeses that you can change that are likely to influence the results. But really have no idea of how you're actually going to do compared to practices elsewhere. One useful way to put findings from measures into context is through the use of benchmarking. The goal of benchmarking is simply to compare your own performance to what's actually achieved by some subset of high performing entities or units, either within larger organization or between organizations. Benchmarks may be set at a specific level. Such as a top 25% or even the top 10%, as a threshold to reach to achieve some sort of recognition, added payment or to simply encourage improvement. Studies have shown that simply sharing relative of rankings or showing practices the 90th percentile benchmark of performance on measures between clinical practices can spur improvement within those groups with lower rankings. In order to effectively and fairly benchmark, comparisons need to be made with other healthcare organizations with similar patient populations, similar types of services delivered, similar qualifications of professionals delivering the services, as well as other characteristics. For example, it would not be useful or fair to compare unadjusted mortality rates of hospitals caring for a largely poor population, with a high proportion of admissions through the emergency room, with the mortality rate of a hospital caring for a very affluent population, with a high proportion, of elective admissions. In order to make a fair and valid comparison in these kinds of situations, some sort of risk adjustment needs to be done, to remove variation that is not directly attributable, to the safety or quality issues within the organization. Benchmarking is only useful if, as the saying goes, you can compare apples to apples. Benchmarking is also not a one or two time effort. It's important to trend the measures related to a benchmark over time. Trending is important to follow the progress or lack of improvement. Most improvement takes time, and being able to see the impact of an intervention on narrowing the gap between your metric and the benchmark over time is absolutely critical. Trend data can tell you whether you are on the right track for the intervention, or you need to make an adjustment. There are several ways to display data on chart that makes it easy to see progress towards a goal. External benchmarking can be done through a number of different sources including organizations that a credit healthcare institutions, organizations that have recognition or other kinds of public measurement programs, or using measures based on data submitted to government agencies. Within the US, there are organizations committed to quality and benchmarking, such as Leapfrog Group, that offers competitive benchmarking reports to hospitals, NCQA, which has benchmarks for its measures, and specialty organizations which have established benchmarks for their own particular specialties and areas of care. There's a number of efforts to create the ability to compare measures across countries, including work done by the Commonwealth Fund and the World Health Organization. As a resource, the Rand Corporation has an excellent summary of projects exploring the feasibility of comparing measures among different countries. Some initial issues that have emerged include, concern about the lack of consistency and the specification measures, and comparing institutions with differing governing structures, payment systems, and regulations. Comparing quality measures across countries is especially challenging, but not impossible. And there is much to learn from one another. For those of you who have an interest in what is being done related to international measure comparisons, information about the Rand study is included in the resources for this module. The word, benchmarking is also sometimes used to indicate a systematic search for and adoption of best practices with the intent of bringing everyone to the highest level possible. Used in this latter sense, benchmarking can be traced back to the 1930s, but became much more widely integrated into business practices when it was adopted by Xerox in 1979. Xerox controlled the market for copiers until the late 1970s, when competitors reduced the Xerox market share down to 35%. In order to become more competitive, external benchmarking was introduced by Xerox, and compared metrics related to all steps in their production process to competitors, and found for example that they could reduce their number of suppliers, from 5,000 to 300 and cut costs significantly. Benchmarking to best practices in this sense is a fairly recent innovation in healthcare spurred by the Institute for Healthcare Improvement and others. Although aspects of it are inherent in accreditation activities, such as those of the joint commission or NCQA. So you've selected a problem. You've confirmed from benchmarking that you indeed have a problem and you're ready to tackle that problem. What do you do now? In the next episode we'll talk more about systematic approaches to quality improvement, focusing on defining interventions and evaluating those interventions. See you in the next episode.
