The goals and rationale of a clinical practice guideline (CPG) or systematic review (SR) are to improve treatment based on current best evidence, optimize patient care through treatment decisions designed to improve the quality and efficiency of care, and identify areas for future research in which evidence is lacking.

AAOS recognizes that only high-quality CPGs and SRs are trustworthy and goes to great lengths to ensure that the integrity of evidence analysis is maintained during the guideline-development process. Multidisciplinary clinician workgroups and AAOS staff methodologists work together to synthesize published research to provide transparent summaries of research findings for each specific orthopaedic disease or condition.

Currently, AAOS has 18 CPGs containing more than 400 recommendations, all housed in the OrthoGuidelines app. The CPGs provide evidence-based recommendations that span entire episodes of care.

To maintain the scientific integrity of the recommendations, all AAOS guidelines meet the Appraisal of Guidelines for Research and Evaluation (AGREE) II criteria. The AGREE instrument was established in 2003 by a group of international guideline developers and researchers to address the variability in guideline development, and it concentrates on guideline methodology and transparency issues. AGREE defines guideline quality as “the confidence that the potential biases have been adequately addressed and that the recommendations are both internally and externally valid and are feasible for practice.” AGREE is a reliable and useful tool for the assessment of guidelines, and the purpose is to provide a framework for assessing the quality of guidelines, provide a methodological strategy for the development of guidelines, and inform what information and how information should be reported in guidelines. AAOS likewise upholds strict Institute of Medicine standards for CPG development. For more information on the standards, visit The National Academies of Sciences, Engineering, and Medicine website.

Frequently, AAOS receives requests from external organizations to endorse their CPGs or SRs. Recognizing that guidelines can vary in quality and in an effort to ensure confidence in guidelines’ resulting recommendations, the Committee on Evidence-based Quality and Value (EBQV) established the “Specialty Society Clinical Practice Guideline Standard Criteria for Endorsement Consideration” checklist, which provides a set of standard criteria that all externally developed musculoskeletal CPGs must meet prior to requesting AAOS guideline endorsement.

Featuring the six domains of CPG quality, along with the 23 core items, the endorsement checklist covers all aspects in the guideline-development process as defined by AGREE II. The domains of quality are:

• scope and purpose
• stakeholder involvement
• rigor of development
• clarity of presentation
• applicability
• editorial independence

The following information provides a brief overview of the domains and associated core items.

Domain 1: scope and purpose contains core items one through three. This section appraises the overall goal of the CPG. The following questions should be answered: Are the overall objectives of the guideline precisely described? Does the guideline specifically state its foreseeable health benefits to the clinical situation, as well as clearly define its intended population?

Domain 2: stakeholder involvement contains core items four through six. The second section focuses on the individuals who were involved in the guideline-development process. Main concerns in this segment are: Does the guideline disclose the composition of the development group, as well as its relevant expertise? Are the guideline’s targeted users clearly defined?

Domain 3: rigor of development contains core items seven through 14. In this segment, the process used to gather and analyze the evidence is examined. Does the guideline provide the sources consulted for the literature search (e.g., PubMed, Embase, Cochrane Library), as well as the search strategy used to gather the resulting information? The literature-search strategy needs to be comprehensive and free from all potential biases. Are the methods used to formulate the guideline recommendations clearly described, and is the resulting strength of evidence defined? Are the benefits and potential harms of the guideline’s recommendations clearly listed within each recommendation? Has the guideline been externally vetted by content experts prior to its publication (e.g., peer review, public comment)? Are the guideline’s revision plans clearly stated within the resulting document (e.g., revised with new evidence, emerging treatment options, new technology; reissued; withdrawn after a specified time period)?

Domain 4: clarity of presentation contains segments 15 through 17. This domain deals with the language and composition of the guideline’s recommendations. Are all recommendations unambiguous, clearly presented, and easily defined?

Domain 5: applicability contains segments 18 through 21. In this domain, the feasibility and applicability of the guideline’s recommendations are addressed. Does the guideline describe who would be implementing the stated recommendations, as well as define any obstacles that may impact the execution of any of the guideline’s recommendations? Does the guideline provide any tools or advice on how to put the guideline recommendations into place? Are there any resources or implications for applying stated recommendations?

Domain 6: editorial independence contains segments 22 and 23. The final domain deals with potential conflicts of interest and biases. Does the CPG clearly state the funding body, and does it notate whether any funding was received from an outside commercial source to support development of the resulting document? If external funding has been obtained, is there a statement noting that the outside source has not influenced the final recommendations? Has the guideline-development group disclosed any potential conflicts of interest, and have these competing interests been recorded and addressed?

Even if an external guideline fulfills all the listed criteria, eligibility for review does not guarantee endorsement, as the guideline must still go through an approval process. That process includes approval by the Committee on EBQV, Council on Research and Quality, and AAOS Board of Directors. The endorsement process ensures that all AAOS-endorsed guidelines are rigorously developed and present trustworthy findings.

For more information on the Specialty Society Clinical Practice Guideline Standard Criteria for Endorsement Consideration process, contact Barbara Krause at krause@aaos.org.

Barbara Krause is the quality improvement specialist for clinical quality and value at AAOS.

Reference 

1. AGREE Collaboration: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 2003;12:18-23.