At their spring meeting, members of the AAOS Committee on Evidence-Based Quality and Value Committee (EBQVC) made significant changes to the processes and language of the AAOS Clinical Practice Guidelines (CPGs). Developed to enhance, improve, and increase the value of the CPG program, the new processes and language are based on member input and peer review.
Since the introduction of CPGs in 2007, the CPG development process and recommendation language have not changed much, and the committee believed that changes were necessary to clarify specific areas within the development and reporting process.
The AAOS CPG program is a mature, respected surgical evidence-based guideline process. The initial methodology was developed from technology- and pharmaceutical-based protocols and was often difficult to apply to the varied treatments that orthopaedists offer.
The changes adopted should improve the clinical relevance and clarity of the CPGs, simplify the process, and enable the reporting of the evidence-based CPGs in a more natural and understandable format.
The first process change will occur during the introductory meeting of the CPG work groups. Instead of drafting preliminary recommendations, as has been done to this point, the work group will develop a PICO question to guide the literature search.
A PICO question is a research question that includes the following parameters:
- outcome and time
These parameters provide further clarity in defining inclusion criteria for the literature review and evaluating the evidence. For example, in the AAOS CPG on Pediatric Developmental Dysplasia of the Hip, a preliminary recommendation may previously have been stated as “We recommend that children with a positive instability exam at age X be treated with Y.” A PICO question format would define and clarify this statement as follows: “Do infants between birth and 6 months of age with a positive hip instability exam who are treated with a brace have positive outcomes? (Define outcomes.)”
CPG work group consensus-based recommendations are determined by expert opinion of the guideline work group and reflect areas that have no available supporting evidence. Initially, consensus-based recommendations were issued only when the service in question had virtually no associated harm and was low cost (eg, a history and physical exam) or when not establishing a recommendation could have catastrophic consequences.
The EBQVC has now decided that a consensus-based recommendation is permissible only when not establishing a recommendation could have catastrophic consequences. The additional qualifiers of “low cost” and “no associated harm” were omitted due to the difficulty in creating universal definitions that could apply to a multitude of conditions.
Research quality designations
The EBQVC also highlighted the difficulty in grading the quality of literature associated with surgery research. High-level randomized controlled trials depend both on clinical equipoise—an acceptance by the clinical community that both options are appropriate—and on a surgeon’s individual equipoise—the surgeon’s belief that a patient might benefit equally from either option. Because this is difficult to achieve, few high-strength recommendations can be developed. (See “Can You Admit to Being Unsure?” AAOS Now, July 2009.)
The AAOS CPG policy has always allowed research articles to be adjusted upward if the research is of high applicability or if providing the intervention decreases the potential for catastrophic harm, such as loss of life or limb. Now, these criteria have been expanded, based on the Grading of Recommendations Assessment, Development, and Evaluation methodology.
The quality of the assessment of specific research may be adjusted by the work group based on the following:
- The study has a large (> 2) or very large (> 5) magnitude of treatment effect; this criterion is used for nonretrospective observational studies.
- All plausible confounding factors would reduce a demonstrated effect or suggest a spurious effect when results show no effect.
- Consideration of the dose-response effect.
The EBQVC also decided to continue to use the “best-available evidence synthesis” currently employed in AAOS CPGs. The best-available evidence synthesis determines the strength of the rating of recommendations. It is based on those articles that meet the inclusion criteria for each recommendation and are rated as the highest quality evidence.
However, this process excludes many articles because they are not “best-available evidence.” For example, if the rating is based on high-quality articles, then lower quality articles are excluded because they are not the “best-available evidence.”
The EBQVC decided to summarize all articles that meet the inclusion criteria for each recommendation. All articles that meet the inclusion criteria but do not meet the best-quality evidence level will be identified in an appendix and acknowledged as relevant to a particular recommendation within the CPG document.
Rating categories and language
The EBQV Committee made significant changes to the number of categories of recommendations and the language used to describe them (Table 1).
The often-confusing “Inconclusive” recommendation was eliminated. Those recommendations will now be included within the “Limited” category. The committee believes this change better represents the clinical utilization of evidence-based medicine.
The language used to present the recommendations was changed as well. Instead of using language that reflects the work group (ie, “We recommend…”), the recommendations language will now reflect the evidence (ie, “Strong evidence supports that…”). This language also presents the findings of the work groups in a more practical manner.
In addition, the recommendation statement will now include the rating of the strength of recommendation and the basis for the determination (ie, lack of efficacy, increased harm) within it.
Lastly, to increase the practicality and applicability of CPG recommendations, the EBQVC decided to include information about the amount of patient counseling time, the use of decision aids, and the impact of future research for each recommendation.
The strength of a recommendation has an impact on each of these areas. A recommendation based on consensus, for example, requires the most patient counseling time and a significant emphasis on using decision aids. Decision aids are less important if the intervention has a strong recommendation supported by high-quality studies (Fig. 1).
Similarly, future research on a topic might have the greatest impact on recommendations supported by limited evidence or consensus. But recommendations currently supported by strong evidence are unlikely to change based on additional research.
As with all approaches to care within orthopaedics, evidence-based medicine and AAOS CPGs will continue to grow, change, and improve to meet the needs of our patients and members. The quality of evidence continues to improve, furthering orthopaedic quality efforts. The EBQVC continues to reinforce that AAOS CPGs are not rigid protocols, but rather syntheses of evidence to be correlated with clinician experience and expertise and patient values.
AAOS Clinical Practice Guidelines can be accessed at www.aaos.org/guidelines
Disclosure information: Dr. Jevsevar—Medacta USA; Omni Life Sciences.
David S. Jevsevar, MD, MBA, chairs the AAOS Committee on Evidence-Based Quality and Value.
- The AAOS Evidence-Based Quality and Value Committee (EBQVC) has made significant changes to the processes and language of the AAOS Clinical Practice Guidelines (CPGs).
- Consensus-based recommendations will be permitted only when not establishing a recommendation could have catastrophic consequences.
- All articles that meet the inclusion criteria—regardless of whether they meet the best-quality evidence level—will be identified and acknowledged as relevant to a particular recommendation.
- New language, fewer categories of recommendation, and additional information will make AAOS CPGs more practical for clinical use.