Every day, orthopaedic surgeons and their patients are faced with making decisions about treatment options that, unfortunately, often don’t have strong evidential support. In addition, what might be considered appropriate care for a 35-year-old generally healthy patient might not be appropriate for a 70-year-old patient with multiple comorbidities.
Clinical practice guidelines (CPGs) may provide some direction. But their strong reliance on evidence may mean that recommendations for many generally accepted treatments are labeled “weak” or “inconclusive” because high levels of evidence may not be—and may never be—available to support their effectiveness compared to other treatments.
As a result, many medical specialties—including the AAOS—have begun developing appropriate use criteria (AUCs) to determine the appropriateness of select procedures. An “appropriate” procedure is one for which the expected health benefits exceed the expected adverse consequences by a sufficiently wide margin. The goal of AUCs is to improve patient care and obtain the best outcomes while considering the subtleties and distinctions necessary in making clinical decisions.
Evidence-based information, in conjunction with the clinical expertise of physicians from multiple medical specialties, will be used to develop the Academy’s AUCs. In turn, the AUCs will also be used to develop Orthopaedic In-Training Examination questions, educational webinars, content for Orthopaedic Learning Center courses, and informative displays at the AAOS Annual Meeting.
This figure shows the process for developing AUCs. Opportunities for AAOS member involvement occur throughout the process and are indicated as dark shaded boxes.
The first step in creating an AUC is to select a topic (Step 1). AUC topics are derived from AAOS CPGs that establish the effectiveness of various procedures for a given disease, disorder, or condition. The AAOS AUC Committee works in conjunction with the Guidelines Oversight Committee and the Evidence-Based Practice Committee, with input from the Board of Directors, in determining potential topics to address with an AUC. The AUC Committee makes the final determination of AUC topics.
AUC topics are carefully selected based on the following four parameters:
- The procedure is widely and frequently performed.
- The procedure is associated with a substantial amount of morbidity and/or mortality.
- The procedure consumes significant resources.
- The procedure has wide geographic variations in use.
After the topic has been selected, the AUC Committee seeks individuals to participate in each of the three panels (Writing, Review, and Voting) involved in the development of the AUC (Step 2). AAOS fellows and members of orthopaedic specialty societies may nominate themselves or other orthopaedic surgeons for each of the panels.
The Writing Panel is a group of 6 to 10 clinicians who are experienced in the procedure under study. Writing Panel members must have completed their disclosures in the AAOS Orthopaedic Disclosure Program; however, conflicts of interest will not affect appointment to this panel.
The Writing Panel is primarily responsible for creating the content for AUCs (Step 3). Through teleconferences and emails, panel members discuss AUC methods, select clinical indications, write definitions and assumptions, approve scenario matrix and literature reviews, and approve modifications suggested by the Review Panel.
AAOS staff develop a literature review by updating and supplementing the literature review that was developed for the corresponding CPG (Step 4). The literature review will be used by the Voting Panel to complement their expertise and experience as they develop the appropriateness ratings.
Review and approval process
The Review Panel is a group of 10 to 30 clinicians who are specialists in the selected procedure. Review Panel members must have completed their disclosures in the AAOS Orthopaedic Disclosure Program; however, conflicts of interest will not affect appointment to this panel.
After the Writing Panel develops the indications, the Review Panel receives the matrix of clinical scenarios, the list of definitions and assumptions, and the literature review (with “mapped” evidence) (Step 5). The panel members then review the materials to ensure that they are representative of patients and scenarios likely to be encountered in a clinical setting. Review Panel members can then suggest that the Writing Panel make changes or modifications in the indications, scenario matrix, definitions and assumptions, or the literature review.
The Writing Panel makes edits (Step 6) based on the Review Panel’s suggestions. After all edits have been made, the Writing Panel approves the final list of indications, definitions, and assumptions. The draft AUC then goes through two rounds of voting.
The Voting Panel is a multidisciplinary group of 10 to 15 clinicians. All Voting Panel members must have completed their disclosures in the AAOS Orthopaedic Disclosure Program. Less than 50% of the AUC Voting Panel may consist of individuals with relevant financial and nonfinancial conflicts of interest. The AAOS Board of Directors and Committee on Outside Interests have established specific conflict-of-interest standards that apply to all members of the Voting Panel for AUCs. These policies are intended to minimize the perception of bias and ensure the integrity of AAOS AUCs. Following an assessment of conflicts of interest, the members of the AUC Voting Panel are appointed from the pool of applicants.
To reduce bias, less than 50 percent of the Voting Panel members should be surgeons who perform the procedure under consideration. The remaining members of the panel should be orthopaedic surgeons, physicians, and other healthcare professionals who are not considered experts on the topic but who would be involved with patients who may have this condition or need this procedure. For example, a physician who might refer the patient to an expert in this area could serve on the Voting Panel.
The Voting Panel is primarily responsible for rating the scenarios that comprise AUCs via two separate rounds of voting. During the first round, which lasts approximately 1 month, panel members individually rate the scenarios created by the Writing Panel via an electronic survey (Step 7a). Treatments for each scenario are rated on a 1 to 9 scale, as shown in Table 1.
The second round of voting takes place after a face-to-face discussion to consider the differences in the results of the first round appropriateness ratings (Step 7b). After this discussion, Voting Panel members vote again and may change any of their Round One appropriateness ratings.
The final AUC document is reviewed and approved by the AUC Committee, the Council on Research and Quality, and the AAOS Board of Directors (Step 8). After the AUC has been approved, the AAOS department of research and scientific affairs will publish it on the AAOS website and begin the dissemination process with an official press release. The AUC will also be released as an interactive web-based application and a summary will be published in the Journal of the AAOS, AAOS Now, and The Journal of Bone and Joint Surgery.
For more information about AAOS AUCs, including frequently asked questions, AUC development procedures and processes, and AUCs currently in development, visit www.aaos.org/auc
Jayson Murray is the manager, appropriate use criteria, in the AAOS department of research & scientific affairs. He can be reached at firstname.lastname@example.org
About the method
AAOS has adopted the RAND/UCLA Appropriateness Method to synthesize the scientific literature and expert opinion on healthcare topics. This method combines the best available scientific evidence with the collective judgment of physicians to determine the appropriateness of performing a procedure. To learn more about the RAND/UCLA Appropriateness Method, go to http://www.rand.org/health/surveys_tools/appropriateness.html
First AUC under development
The topic of the first AAOS AUC is Treatment of Distal Radius Fractures (DRF). Members of the DRF Voting Panel will be meeting soon. The second AAOS AUC, Rotator Cuff Management, is in the beginning stages of development.
Future AAOS AUC topics include the following:
- Knee Arthroscopy
- Imaging for Shoulder Pain
- Operative Treatment of Hip Fractures
- Total Knee Replacement
- Imaging for Knee Pain
- Imaging for Low Back Pain
- Lumbar Fusion for Low Back Pain
- Total Hip Replacement
AAOS fellows who are interested in participating on any of the panels for any of these topics should contact Jayson Murray, AUC manager, at email@example.com