Position Statement

Public Reporting of Provider Performance

This Position Statement has been developed as an educational tool based on the opinion of the authors. It is not a product of a systematic review. Readers are encouraged to consider the information presented and reach their own conclusions.

Background

Performance measurement is a growing phenomenon in which providers of health care are required to report the results of clinical measures of quality. There has been increasing demand to accelerate quality improvement and promote transparency through public reporting of this performance data.¹⁰ This trend was accelerated by the passage of the Patient Protection and Affordable Care Act (PPACA), which includes provisions to support public reporting. Public reporting has been accompanied by efforts to reduce rising healthcare costs, create accountable and high performing health systems, and improve the overall quality of patient care.¹⁰ Public reporting is viewed as a necessary building block in the pursuit of value-based healthcare, but there remain significant challenges to the accurate collection and reporting of information on the cost, quality, and value of healthcare. These challenges include accurate collection of data, appropriate risk adjustment for patient comorbidities, adequate sample size, validity, reliability, and clinical relevance.¹¹

Public reporting would ideally be the public dissemination of statistically valid, reliable, and useful information^{1, 17} of performance data that can be used by consumers, providers, purchasers, health plans, and policymakers.⁷ A number of metrics, including process measures, volume measures, outcome measures, structural measures, and patient experience, have been used to measure and report healthcare physician and hospital performance.

The American Association of Orthopaedic Surgeons (AAOS) supports public reporting as a means to empower patients and increase transparency and accountability of providers. It is critical that the measures reported be reliable, actionable, meaningful, and appropriately risk adjusted.

Development of Public Reporting Programs

Performance profiling did not figure prominently in U.S. health care policy until 1986, when the Health Care Financing Administration (now the Centers for Medicare and Medicaid Services, or CMS) began to publicly report hospital specific mortality rates.¹⁶ More recently, efforts have been made to use administrative billing data to create public reporting programs. Most notably, CMS developed a large public reporting program known as Hospital Compare (http://www.hospitalcompare.hhs.gov), measuring and reporting process measures of high-quality care for acute myocardial infarction, heart failure, pneumonia, and general surgery.¹⁶ CMS, through its Physician Quality Reporting System (PQRS), and hospital reporting of the Surgical Care Improvement Project (SCIP) measures, has also expanded its quality reporting and public reporting initiatives. To date, these are primarily measures of process (such as a patient receiving a treatment within the right time). Several of these CMS initiatives have specific financial incentives for participation and penalties for non-participation. ¹⁵ Other organizations such as the National Committee for Quality Assurance (NCQA), Healthgrades, Inc., Leapfrog, and Blue Cross Blue Shield use public reporting as a window into health outcomes.

The AAOS believes that systems for measuring and reporting quality in health care should continue to evolve and expand. The current generation of quality measures, which primarily rely on process measures and administrative data, have not yet been proven to accurately correlate with improved functional outcomes, which are the primary outcomes of interest to patients who undergo orthopaedic procedures. The AAOS strongly supports development of patient centered outcome measures and patient centered outcomes reporting with a goal of increasing public awareness of the relative risks, benefits, and costs associated with operative and non-operative care of musculoskeletal conditions. The AAOS also believes public reports of provider performance should be thoroughly vetted for accuracy and have a reliable and rapid mechanism for challenging and removing inaccurate information from provider profiles.

Effects of Public Reporting on Quality Improvement and Physician Performance

The effect of public reporting on health care outcomes is a subject of much debate. While public reporting can improve accountability and transparency, there is some risk that publication of complication rates will cause physicians to avoid taking on more difficult patients (e.g. “cherry-picking”) in an attempt to decrease their risks of complications and thereby improve their quality ranking and achieve "target rates" for health care interventions. This may result in decreased access to care for certain vulnerable populations of patients who are at higher risk for peri-operative morbidity.

In the past, public reporting and risk adjustment have been largely based on administrative or claims data. Administrative data are routinely collected, relatively inexpensive to analyze, and allow for easy identification of geographical and ethnic subgroups with particular access problems.¹ However, they do not address the nuances of comorbidities, severity, conditions that were present on admission, complications, patient satisfaction, patient education, and provide inadequate risk adjustment. Clinical data is accurate and comprehensive, but it is very expensive and often difficult to obtain as there are variations in how hospitals and physicians document and collect data. With the substantial differences in the cost of obtaining various types of clinical data, enhancement of administrative data sets appear to be both practical and desirable.¹⁴ It is important to collect and publicly report meaningful information; however, there is a need for harmonization between administrative and clinical data because both have pluses and minuses. Administrative data is accessible, but is a blunt tool. Harvesting clinical data gives more accurate information, but places a significant work burden on physicians.

As physicians, we recognize there can be vast differences between groups of patients with the same diagnoses. Diabetes can be mild, controlled, brittle, or uncontrolled. The variations are difficult to measure and have significant implications on surgical decision making. Other conditions, such as a history of previous fracture in a patient undergoing knee replacement can make the surgery markedly more complicated, yet are difficult to accurately convey to public reporting systems. Further, there is growing evidence that patients’ socioeconomic status has a profound influence on outcomes.⁵

The AAOS reinforces the need for balance in implementing public reporting systems between the increasingly urgent need to improve the quality and efficiency of care and the importance of developing clinically valid and appropriately risk-adjustment performance measures that will ensure ongoing access for patients who are at higher risk of complications and poor outcomes. The AAOS supports ongoing research into performance measure development and proper risk adjustment of all publically reported outcome measures. Risk adjustment for age, sex, comorbidities, disease severity, and socioeconomic status allows measures to be believable and comparable across providers and delivery systems.

Summary

Public reporting of provider performance is rapidly expanding. The AAOS supports the judicious use of public reporting that is clinically relevant, timely, valid, reliable, appropriately risk adjusted, and actionable, and that minimizes the burden of data collection on patient, physicians, and hospitals. In order to achieve these goals, the AAOS believes it is essential for physicians and public reporting agencies to work collaboratively to:

• Develop clinically valid and appropriately risk-adjusted performance measures in order to ensure ongoing access for patients who are at higher risk of complications and poor outcomes
• Continue research into performance measure development and proper risk adjustment of all publicly reported outcome measures
• Create comparable measures that take into account age, sex, comorbidities, disease severity and socioeconomic status
• Fully vet all public information to ensure it is accurate, clinically relevant, timely, valid, reliable and actionable by the public and care providers

References:

1. Asch R, Sloss E, Hogan C, Brook R, Kravitz R: Measuring underuse of necessary care among elderly Medicare beneficiaries using inpatient and outpatient claims. JAMA. 2000; 284(18):2325-2333.
2. AQA Principles for Public Reports on Health Care. 2006.
3. Bhattacharyya T, Freiberg A, Mehta P, Katz J, Ferris T: Measuring the report card: The validity of pay-for-performance metrics in orthopedic surgery. Health Affairs 2009; 28:526-532.
4. Brook R, Leatherman S, Marshall M, Shekelle P: The public release of performance data: What do we expect to gain? A review of the evidence. JAMA. 2000; 283(14):1866-1874.
5. Butler R, Rosenzweig S, Meyers L, Barrack R: The impact of socioeconomic factors on outcomes after THA. Clin Orthop Relat Res (2011) 469:339–347.
6. Casalino L, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D: Will pay-for-performance and quality reporting affect health care disparities? Health Affairs, 26, no.3 (2007):w405-w414.
7. Cleverly W, Cleverly J: Is there a cost associated with higher quality?” Health Financial Management, January 2011; 65, no.1; pages 96-102
8. Colmers J: Public reporting and transparency. The Commonwealth Fund, February 2007.
9. Hibbard J, Sofaer S: Best practices in public reporting no. 1: How to effectively present health care performance data to consumers. Rockville, MD: Agency for Healthcare Research and Quality; June 2010. AHRQ Publication No. 10-0082-EF.
10. Lagu T, Lindenauer P: Putting the public back in public reporting of health care quality. JAMA 2010; 304(15):1711-1712.
11. Lindenauer P, Remus D, Roman S, Rothberg M, Benjamin E, Ma A, Bratzler, D: Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96.
12. Marjoua Y, Butler C, Bozic K: Public Reporting of Cost and Quality Information.
13. Marshall M, Shekelle P, Brook R, Leatherman S: Dying to Know: Public Release of Information about Quality of Health Care. Santa Monica, CA: RAND Corporation, 2000.
14. Marshall M, Romano P, Davies H: How do we maximize the impact of the public reporting of quality of care? Int J Qual Health Care (2004) 16 (suppl 1): i57-i63.
15. Marshall M, Shekelle P, Davies H, Smith P: Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003 May-Jun; 22(3):134-48.
16. The Patient Portability and Affordable Care Act, Sections 3002, 3007, 10331. 2010.
17. Pine M, Jordan H, Elixhauser A, Fry D, Hoaglin D, Jones B, Meimban R, Warner D, Gonzales J: Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA. 2007; 297(1):71-76.
18. Ross J, Sheth S, Krumholz H: State-sponsored public reporting of hospital quality: Results are hard to find and lack uniformity. Health Affairs, 29, no.12 (2010):2317-2322
19. Shaller D, Sofaer S, Findlay S, Hibbard J, Lansky D, Delbanco S: Consumers and quality-driven health care: A call to action. Health Affairs, 22, no.2 (2003):95-101.
20. Shoemaker W: “Benchmarking boon: Tapping publicly available data to improve performance”. Health Financial Management, June 2011, 65, no. 6; pages 88-94.
21. Tang P, Ralston M, Arrigotti M, Qureshi L, Graham J: Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: Implications for performance measures. J Am Med Inform Assoc 2007; 14(1):10-15.
22. Werner R, Asch D: The unintended consequences of publicly reporting quality information. JAMA. 2005; 293:1239-1244.

February 2012 American Academy of Orthopaedic Surgeons.

This material may not be modified without the express written permission of the American Academy of Orthopaedic Surgeons.

Position Statement 1183

For additional information, contact the Public Relations Department at 847-384-4036.