PCPI: What you should know about Consortium performance measures

Director, Clinical Performance Evaluation
American Medical Association Chicago, IL

Measures for improvement have learning as their goal. They are the videotapes we watch after the game…the stop watches of the coaches who help us with our stride.”¹ So said Donald Berwick, MD, a leader in clinical quality improvement and patient safety.

Clinical performance measures indicate whether, or to what extent, a process of care (eg, prescribing antiplatelet therapy for patients with coronary artery disease) or outcome of care (eg, visual acuity following cataract surgery) occurs.

You may have been using clinical performance measures for years, in local quality improvement efforts or as part of national programs such as the American College of Surgeons’ National Surgical Quality Improvement Program or the Society of Thoracic Surgeons’ National Database. Or you may have just experienced your first taste of performance measures through the Centers for Medicare & Medicaid Services (CMS) Physician Quality Reporting Initiative (PQRI), launched in the summer of 2007, or through a new American Board of Internal Medicine Practice Improvement Module for Maintenance of Board Certification.

However long you may have been pursuing performance measurement and improvement, you would do well to understand how the measures were developed and the intent of the developer of the measure. Physicians should satisfy themselves that the measures are credible: Are they grounded in clinical evidence? Are the measures relevant for clinical practice? Do they take into account patient preferences?

  What is the intent of the Consortium?

The Physician Consortium for Performance Improvement ^® (Consortium), convened by the American Medical Association, has more than 100 representatives from national medical societies, state medical associations, and medical specialty boards. It has developed performance measures in 30 clinical areas (TABLE 1).

The intent of the Consortium is to develop measures that are useful to physicians at the point of care (eg, a reminder to consider prescribing antiplatelet therapy or a reminder that this patient should not receive antiplatelet therapy because she recently experienced gastrointestinal bleeding) and in practice-wide analysis (eg, how many of my patients with diabetes received an eye exam this year?).

30 clinical topics for which the Consortium has developed measures

How measures are developed

The Consortium’s approach to measure development includes the following steps:

1. Identifying opportunities for improvement

2. Involving representatives from all medical specialties and other relevant health care disciplines in the process

3. Linking measures to an evidence base

4. Supporting clinical judgment and patient preferences

5. Testing measures

6. Promoting a single set of measures for widespread use and multiple purposes.

  Looking at the areas of greatest need

In selecting topics for performance measure development, the Consortium places a priority on areas where deviations in care or gaps in care are prevalent. And the net is cast widely to look for areas of under-use (failing to provide an effective service when it would have produced favorable results) and overuse (providing a health service when its risk of harm exceeds its potential benefits).²

For example, according to guidelines developed by the National Kidney Foundation, a functioning AV fistula is the preferred method for hemodialysis. Compared with other dialysis access modalities, AV fistulae have the lowest rates of thrombosis and infection, and as a result the costs of AV fistulae and maintenance are the lowest.³ Yet among 18 end-stage renal disease (ESRD) networks in a national study, the percentage of ESRD patients dialyzed with the recommended AV fistula varied from 28% to 56% for prevalent patients, and from 22% to 61% for incident patients.⁴

The Consortium also looks among subpopulations to identify disparities in care. For example, men are more likely than women to receive the recommended AV fistula, and whites are more likely than blacks to receive an AV fistula.⁴

In response to these findings, the Consortium, in collaboration with the Renal Physicians Association, developed this performance measure for physicians managing the care of ESRD patients: The percentage of patients with ESRD and receiving hemodialysis who have a functioning AV fistula or are referred for an AV fistula/permanent vascular access.

  Deviations and gaps in care

Data on deviations in care and gaps in care are found in the published literature, the Dartmouth Atlas, and the National Healthcare Quality Reports of the Agency for Healthcare Research and Quality (AHRQ). Such data, however, are not available for most clinical topics. When no data are available, measures may be based on expert opinions that gaps or deviations exist. Widespread use of the measures, over time, will validate or invalidate the claim.

  You have a say in measure design

Consortium measures are designed for any physician managing the care of a patient with a particular condition; that is, the measures are not specialty-specific, but rather focused on the patient and her condition or the procedure she received. Therefore, practitioners of all relevant disciplines of medicine—as well as other health care professionals for whom the care topic is within their scope of practice—are included in each measure work group.

Work group representation

Every group also must include actively practicing physicians to whom the measures will apply. Representatives from consumer groups, private health plans, and CMS are invited to participate in work groups to add their perspectives, as are experts in methodology and medical coding. Moreover, each set of draft measures is posted on the internet for a 30-day public comment period. Please see www. physicianconsortium.org for opportunities to comment on draft measures.

  The evidence base for measures: black, white, and gray

David Eddy, MD, PhD, known to many as the author of a seminal paper on the role of guidelines in medical decision making, defines evidence-based medicine (EBM) as “…a set of principles and methods to ensure that, to the greatest extent possible, population-based policies and individual medical decisions are consistent with evidence of effectiveness and benefit.”⁵

Developing these principles, or clinical guidelines, involves a rigorous, systematic review and grading of the quality of individual studies. For a thorough discussion of the guideline development process, see Clinical Practice Guidelines: A Manual for Developing Evidence-Based Guidelines to Facilitate Performance Measurement and Quality Improvement, by Richard Rosenfeld, MD, and Richard Shiffman, MD.⁶

Development of guidelines

The Consortium does not develop clinical guidelines. Rather, it relies on available guidelines to identify aspects of care for measure development. The Consortium’s measure-development work groups prefer to base measures on guideline principles that have the highest level of evidence (well-designed randomized, controlled trials) and the strongest recommendations.

However, since it is generally accepted that less than about 20% of medicine is based on scientific evidence, the Consortium also develops measures when evidence is unavailable but a clear linkage can be made to evidence-based care.⁷

For example, an assessment of a patient’s symptoms and functional status is the first measure in the Consortium’s set of measures for osteoarthritis. It is unlikely that a randomized, controlled trial will be conducted to evaluate the risks and benefits of this assessment; however, a logical link can be made from assessing functional status to evidence-based practices in treating osteoarthritis.

The Consortium always provides clinical guideline statements alongside its performance measures (TABLE 2).

Sample Consortium measure

  Accounting for clinical judgment and patient preference

Another definition at the crux of Consortium work is that of evidence-based medicine by David Sackett, MD: “…the integration of best research evidence with clinical expertise and patient values.”⁸

Physicians exercise clinical judgment and consider patient preferences when making treatment recommendations. Continuing with one of the examples above, a patient with a recent gastrointestinal bleed is not a candidate for antiplatelet therapy. Another patient may be intolerant of the drug. Yet another patient may say he does not want to take a particular drug. Consortium measures provide a means by which physicians can document these valid medical and patient reasons for deviation, as well as system reasons, such as lack of resources or other factors attributable to the health care delivery system.

In calculating a measure, these reasons are known as “exclusions,” because they indicate patients who should not be included in the measure denominator. These reasons, however, should be part of feedback reports to physicians and appear in prompts in electronic health records so that an initial reason for exclusion from the measure denominator, if relevant, can be evaluated and revisited.

  Testing and evaluating measures

Andrew Auerbach, MD, MPH, and colleagues recently argued that interventions to improve the quality of health care should meet the same standards of evidence that are applied to the adoption of new medical technologies.⁹ The performance measures developed by the Consortium are no exception.

Long term, the Consortium will work with health services researchers to understand the impact of performance measurement on patient care and to identify unintended consequences.

Short term, tests of reliability must be completed: Are the same observations collected regardless of who gathers the data or when or where they are gathered?¹⁰ Stephen Persell, MD, David Baker, MD, and colleagues analyzed the validity of Consortium measures within an electronic health record system (EHRS).^11,12 The Consortium must quickly expand its testing portfolio; when available, results or links to tests will be posted on the Consortium Web site.

  Zeroing in on a common set of national measures

Integrating performance measures into physician practice requires time, workflow changes, and resources. These requirements can be doubled or tripled when different organizations define diverse performance measures for the same clinical area. Ideally, physicians would use the same measures for multiple purposes, including participation in a Continuing Medical Education (CME) program, Maintenance of Certification, and participation in a CMS-sponsored initiative. Toward identifying a single, common set of measures acceptable by all stakeholders, the Consortium submits its measures to 2 national organizations positioned to promote a common set of measures: the National Quality Forum (NQF) and the AQA Alliance (formerly the Ambulatory Care Quality Alliance). We encourage physicians to use measures developed by the Consortium, endorsed by the NQF, and selected by the AQA Alliance.

  Measure development is still evolving

Although the Consortium is in its seventh year of measure development, the scope and depth of its work is still nascent. Our “videotapes and stop watches” need to be kept up to date with the latest technological tools. We need to better understand how to integrate performance measures into the fabric of patient care, explore new measures of continuity of care and components of efficiency of care; and uncover unintended consequences of measurement. The process and deliberations of the Consortium will remain open and transparent for physician inspection. Practicing physicians should be aware of the tools the Consortium offers for improving care and can encourage their specialty societies, state medical associations, and medical boards to join the Consortium to help us as we continue to build measures to improve the quality of clinical care.

References

1. Berwick DM. Escape Fire: Designs for the Future of Health Care. San Francisco, CA: Jossey-Bass; 2004:228.
2. Chassin MR. Is health care ready for Six Sigma quality? The Milbank Quarterly. 1998;76(4):565–591.
3.  National Kidney Foundation. K/DOQI Clinical Practice Guidelines for Vascular Access, Update 2006.http://www.kidney.org/professionals/KDOQI/guidelines.cfm. Accessed Oct 10, 2007.
4.  Centers for Medicare & Medicaid Services. 2004 Annual Report, End Stage Renal Disease Clinical Performance Measures Project. Baltimore, MD: HHS, Centers for Medicare & Medicaid Services, Office of Clinical Standards & Quality; Dec 2004.
5. Eddy DM. Evidence-based medicine: a unified approach. Health Aff. 2005;24(1):9–17.
6. Rosenfeld RM, Shiffman RN. Clinical practice guidelines: a manual for developing evidence-based guidelines to facilitate performance measurement and quality improvement. Otolaryngol Head Neck Surg. 2006;135:S1–S28.
7. Booth A, et al What portion of healthcare is evidence based? Resource Guide. www.shef.ac.uk/scharr/ir/percent.html. Accessed Oct 10, 2007.
8. Sackett DL, Straus SE, Richardson WS, et al. Evidence based Medicine: How to Practice and Teach EBM. 2nd ed. Edinburgh: Churchill Livingstone; 2000.
9. Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;257(6):608–613.
10. Cohen BP. Developing Sociological Knowledge. Chicago: Nelson-Hall; 1989:155.
11. Persell SD, Wright JM, Thompson JA, et al Assessing the validity of national quality measures for coronary artery disease using an electronic health record. Arch Intern Med. 2006;166:2272–2277.
12. Baker DW, Persell SD, Thompson JA, et al Automated review of electronic health records to assess quality of care for out-patients with heart failure. Ann Intern Med. 2007;146:270–277.