Table 1 AAOS Evidence to Decision Framework domains
Adapted from: GRADE: Detailed judgments in Evidence to Decision (EtD) frameworks. Available at: https://healthpolicysystems.biomedcentral.com/articles/10.1186/s12961-018-0320-2/tables/3. Accessed March 2, 2021.

AAOS Now

Published 3/24/2021
|
Danielle Schulte, MS

Get to Know the AAOS CPG Evidence to Decision Framework

Each year, AAOS publishes several new and updated Clinical Practice Guidelines (CPGs). To ensure that the CPGs provide the highest-quality guidance to members, staff and member volunteers follow a systematic process of literature review and recommendation creation. Standard to the process is a systematic review of the literature. During such a review, published literature is organized and reviewed by topic, as well as appraised for quality. The evidence is then aggregated and utilized by the CPG workgroup to establish the evidence base for each potential recommendation up for consideration in the CPG. At the core of the CPG process is the principle of evidence-based medicine, which dictates that medical practice decisions be made based on “current best evidence.” Although this tenet is the backbone of the CPG process, it cannot be viewed in a vacuum. Tied to this principle is the idea that “individual clinical expertise” should also be an integral component of such decisions.

To ensure appropriate incorporation of clinician expertise and underscore for readers what contextual considerations must be made when they interpret recommendations, the AAOS Committee on Evidence-based Quality and Value approved the incorporation of an additional protocol in April 2019 for combining evidence and experience into the CPG recommendation creation process, based on the Evidence to Decision Framework (EtDF). The EtDF is a tool authored by the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) Working Group, an organization whose focus is on developing systematic and transparent approaches to assessing and reporting on the quality of evidence included in CPGs. The EtDF tool is a structured method of recommendation creation incorporating published evidence with real-world clinical experience in a systematic way that allows for workgroup transparency concerning the considerations and judgments that go into creating recommendation language. This transparency also allows users to understand the context in which a recommendation was created and make informed conclusions about how best to incorporate a given recommendation into their current practice.

Previously, there was little room in the process to include the nuanced perspectives experts could deliver. With the adoption of the EtDF, this not only becomes more possible but is now actively sought out. Expert opinion can now be included alongside the evidence base to further bolster recommendations and position them as usable guidance rather than just reference statements. Under the new protocol, the EtDF is to be employed by a CPG workgroup after the literature review is completed and the workgroup reconvenes to discuss the evidence. During deliberations, the framework is utilized as the roadmap to organize the group’s thoughts and judgments for each recommendation.

Prior to the EtDF, CPG recommendations were made solely based on the strength of the evidence. Now, in addition to the evidence, physician workgroups follow a structured process for incorporating topic-specific nuance into their decision making and are permitted to represent such considerations to readers by incorporating the information in the recommendation rationale and through upgrading or downgrading a recommendation via a supermajority vote, if the group determines that the additional considerations merit such adjustment.

As the workgroup formulates its recommendations, seven domains are used to focus the conversation and establish the context in which a recommendation should be considered (see sidebar and Table 1).

Although all domains of the EtDF are important to readers, a key facet of this new protocol, in which members have shown interest for some time, is the incorporation of cost discussions within the context of a recommendation. Cost should not be the driver of care, but economic and resource allocation considerations of an intervention cannot be ignored and are continually a main concern for healthcare providers and their patients. Cost is a necessary consideration for a guideline panel to consider so that providers can clearly understand costs depending on each patient’s situation; with the EtDF, AAOS recommendations will now be more useful to clinicians as they adapt to value-based environments.

Furthermore, these EtDF considerations will amplify a guideline’s ability to facilitate patient-clinician communication. Clearly delineating the significant factors to be considered, such as costs, benefits, and harms, before a mode of care is employed, allows the patient and care provider to share the decision-making process. Identifying significant factors can better contextualize what a recommendation truly means and how it should or should not be employed in respective situations.

Ultimately, to create the final recommendations for inclusion in a CPG, the workgroup must evaluate the various considerations of the EtDF and incorporate their own clinical knowledge and experience. This balance of published evidence and expert experience is an important aspect of the recommendation creation process, as it allows the workgroup to make sufficiently considered recommendations that help readers understand how a recommendation was created and therefore better understand why certain determinations were made. This clear delineation of the various considerations along with the published evidence ultimately will allow readers to more clearly interpret what a recommendation will mean for their practice and enable them to provide the best possible care to their diverse array of patients.

For more information on AAOS’ CPGs and methodology, visit orthoguidelines.org and https://aaos.org/quality/research-resources/methodology.

Danielle Schulte, MS, is a clinical quality and value development manager in AAOS’ Department of Clinical Quality and Value.

References 

  1. Sackett DL, Rosenberg WM, Haynes RB, et al: Evidence Based Medicine: What It Is and What It Isn’t. BMJ 1996;312:71-2.
  2. GRADE: Home. Available at: https://www.gradeworkinggroup.org/. Accessed January 28, 2021.
  3. Alonso-Coello P, Schünemann HJ, Moberg J, et al: GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. Gac Sanit 2018;32:166.e1-166.
  4. GRADE: Detailed judgments in Evidence to Decision (EtD) frameworks. Available at: https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-018-0320-2/tables/3. Accessed January 28, 2021.

Seven domains considered in the AAOS Evidence to Decision Framework

The framework utilized by AAOS consists of seven domains within which a Clinical Practice Guideline workgroup will organize the constituent considerations of a recommendation (Table 1):

  1. strength of evidence
  2. benefits and harms
  3. outcome importance
  4. cost-effectiveness/resource utilization
  5. acceptability
  6. feasibility
  7. future research