Published 10/1/2016
David R. Chandler, MD

Measuring Up: The Making of a Performance Measure

At its initial meeting in 2014, the AAOS Performance Measures Committee (PMC), chaired by Warren Dunn, MD, MPH, decided to develop a set of performance measures for management of hip fractures in the elderly. The Committee also decided to steward the Physician Quality Reporting System (PQRS) Measure #109—Osteoarthritis Function and Pain—which means that AAOS now owns the measure, is responsible for maintaining the measure, and AAOS staff serves as the point of contact for parties interested in implementing the measure. The PMC also committed to initiating the development of two performance measure sets per year in 2016 and 2017 (Table. 1).

Khaled J. Saleh, MD, MSc, FRCSC, FACS, serves as the oversight chair for the Osteoarthritis: Function and Pain Assessment work group. The work group was charged with revalidating the existing measure (PQRS #109) as well as drafting a new patient-reported outcome performance measure (PRO-PM). 

W. Timothy Brox, MD, is the oversight chair for the development of performance measures around management of hip fractures in the elderly. He has considerable experience working with this topic, having also chaired the work group for the development of the clinical practice guideline (CPG) on "Management of Hip Fractures in the Elderly," which was adopted by the AAOS Board of Directors in September 2014.

In this clinically important area, an examination of the peer review literature established the existence of performance gaps. "We have evidence of a significant variation in outcomes across different hospitals and care models," said Dr. Brox, "and we need to achieve the best possible results for all of our elderly hip fracture patients."

An open call to stakeholders both within and outside the AAOS was made to establish a technical expert panel. In addition to representatives from orthopaedic specialty societies such as the American Association of Hip and Knee Surgeons and the Orthopaedic Trauma Association, physical therapists, nurses, emergency room physicians, anesthesiologists, and physical medicine and rehabilitation physicians were invited to participate.

Dr. Brox noted that a set of five to eight performance measures will be drafted, based on the hip fracture CPG. These measures will be checked for evidence, validity, importance, quality gaps, and potential for improvement. They will also be validated against a working national hip fracture data set.

"Attribution in this data set is to the hospital level," said Dr. Brox. "Attribution to the physician level is problematic due to variations in hospital management. Additionally, reporting is complicated because numerous providers are involved in managing the hip fracture patient, including the admitting physician, hospitalists, a variety of consulting physicians, and the surgeon." For instance, the data base may indicate that the total joint replacement was performed by a consulting internist rather than the surgeon.

As part of the measure set, the work group also plans to develop posthospitalization care measures. The goal is to have a mixture of process measures, outcome measures, and PRO-PMs. "Creating PRO-PMs in this measure set is difficult," said Dr. Brox, "because no validating database is available. PROMIS [Patient-Reported Outcomes Measurement Information System] measures are not likely to be part of this measures group, but inclusion of PROMIS measures will be advocated in the future. Final drafting and plans for validation will be passed through the AAOS approval process with a target submission date of December 2016."

Dr. Brox acknowledged the challenges in risk stratification and risk adjustment around outcomes. Risk stratification would involve dividing patient data based on their risk profile and then comparing the outcomes within these subsets, whereas risk adjustment would involve adjusting patient outcomes based on their co-morbidities. At this time, the data is being placed on the NSQIP data base and has not been analyzed to determine if risk stratification or risk adjustment would be more appropriate.

"For small hospitals, characteristics of the patient population will significantly affect outcomes," Dr. Brox noted. "The work group is very sensitive to the possibility of inadvertent consequences and specifically wants to avoid creating metrics that would motivate small hospitals to divert patients with hip fractures to large centers." According to Dr. Brox, the work group wants to encourage hospitals to look at the potential for improvements in hip fracture care, rather than just to opt out of hip fracture management.

Specialty involvement, support, and communication
During the AAOS Performance Measures Development Summit in July 2014, Kevin J. Bozic, MD, MBA, then chair of the AAOS Council on Research and Quality, challenged orthopaedic specialty societies to commit resources and develop appropriate performance measures for their specialties.

To facilitate that process, the Board of Specialty Societies (BOS) Quality and Patient Safety Action Fund was established. The fund's purpose is to support BOS member organizations' involvement in developing and implementing quality and patient safety initiatives and related advocacy efforts. Proposals that meet prescribed criteria are eligible for matching funds up to $50,000.

Proposals from specialty societies are considered by a grant review committee, which may recommend funding. Several grants have been approved, and proposals are still being solicited.

As payment reform efforts within the Centers for Medicare & Medicaid Services move forward, interest in performance measures for use by orthopaedic specialists is increasing. An effort to maintain strong communication with specialty societies related to performance measure development and implementation has been initiated through the PMC. This effort is being led by George F. Muschler, MD, and myself. Strategies for soliciting, gathering, and sharing of information between the AAOS PMC and specialty societies—as well as web-based resources and educational articles—will be part of the plan.

Get involved
Participation by AAOS members is encouraged. Opportunities include serving on a measure development work group, submitting a performance measures topic, or becoming a testing site partner. Volunteers to fill vacancies on the PMC will also be needed. More information is available on the AAOS website and the committee appointment program (www.aaos.org/cap).

David R. Chandler, MD, is a member of the AAOS Performance Measures Committee.

Editor's Note: This is one of an ongoing series of articles prepared by the AAOS Performance Measures Committee. Previous articles include "Measuring Up: Defining Performance Measures" (AAOS Now, August 2016) and "Measuring Up: The Performance Measure Development Process" (AAOS Now, September 2016).

Additional Information: