Published 7/1/2008
William C. Watters III, MD

Defining evidence-based clinical practice guidelines

What is an evidence-based clinical practice guideline? To answer that question, we must first examine its components.

A “Clinical Practice Guideline” (CPG) has been defined in a very structured manner by the Institute of Medicine as a “systematically developed statement to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”

“Evidence-based” implies that the document or recommendation has been created using an unbiased and transparent process of systematically reviewing, appraising, and using the best clinical research findings of the highest value to aid in the delivery of optimum clinical care to patients.

Thus, a meaningful answer to the question is that evidence-based CPGs are a series of recommendations on clinical care, supported by the best available evidence in the clinical literature.

Having defined evidence-based CPGs, we now have more questions to address, including the following: Are evidence-based CPGs superior to typical CPGs? How are evidence-based CPGs developed? How are they used, and are they good or bad for me and for my patients?

Superiority of evidence-based CPGs
CPGs have been used and abused for a long time. Early efforts by the AAOS to develop CPGs in the 1990s were intended to support high-quality orthopaedic care and were consensus-driven. In a consensus-driven process, a panel of thought-leaders on a certain topic are brought together, a literature review is done, and a document of recommendations is produced based on the consensus of the review panel.

Although meant to be unbiased “position statements” on best care, these documents were problematic. The development process lacked transparency, and an inherent bias that exists among any group of experts toward their own treatment goals was present. That the very same literature review could be used by a different group (such as industry guideline groups and insurance companies) to derive very different conclusions more consistent with their own goals was proof that such an approach was biased and untenable. Practicing physicians and their patients often felt cheated when care decisions were based on guidelines more oriented to economic, rather than quality, goals.

By applying the techniques of evidence-based medicine in guideline development, the role of opinion—and thus bias—is markedly reduced and the value of the rich scientific literature of clinical medicine is elevated and evaluated in a systematic fashion to provide transparency and minimize bias in evidence-based CPGs. Evidence-based CPGs are thus superior to non–evidence-based CPGs and are true instruments of improved patient care.

Development of evidence-based CPGs
At the AAOS, the Guideline and Technology Oversight Committee (GTOC) is responsible for the administration of all guideline development and technology overviews. Potential guideline topics are submitted to the GTOC, which selects those to be developed and is instrumental in choosing a work group chair for the topic. The chair is responsible for selecting a co-chair and assembling a workgroup of six to eight members from the AAOS and specialty societies to promote diverse viewpoints.

The role of the Evidence-based Practice Committee is to set standards for guideline methodology. Members of the work group are expected to have training in developing, reviewing, and evaluating clinical literature using these standards. At this point, the work group interacts closely with the AAOS Evidence and Analysis groups, headed by Charles Turkelson, PhD, to develop simulated recommendations relevant to the guideline topic. These recommendations consider both benefits and harms and cover all topics of interest. Study inclusion criteria—such as study designs and patient-oriented outcomes—are constructed, along with exclusion criteria—such as fewer than 10 patients or retrospective case review—that would disqualify some studies for consideration.

The AAOS Evidence and Analysis group then searches multiple electronic databases for relevant data, sorts the data, and rates its quality on a per-paper basis into Levels of Evidence, based on the soundness of the research design and its execution. This process results in an evidence report. In a series of joint meetings, the Evidence and Analysis and work groups review the evidence report, grading the levels of evidence available and applying them to recommendations. If adequate evidence is lacking to support a specific recommendation, a consensus statement is developed in a very transparent fashion and under specific rules. All consensus statements are labeled as such in the guideline (Fig. 1).

The final document, which can exceed 600 pages, goes through a peer-review process involving societies other than the AAOS. This “public comment period” is followed by a structured review and approval process within the AAOS that takes the guideline all the way up to the Board of Directors. The finalized guideline is posted on the AAOS Web site, at www.aaos.org/guidelines

The AAOS currently has 11 such guidelines in development.

Use and value of CPGs
The transparent, structured process used to develop CPGs makes them easy to use and difficult to abuse. Evidence-based CPGs are being used to develop quality measures and can be used to support referrals when they are questioned by insurance companies; they also serve as education tools for patients.

On a national level, evidence-based CPGs are a direct means of quality improvement and play an important role in the development of performance measures for pay-for-performance reimbursement programs. The AAOS evidence-based CPGs are giving the practicing orthopaedist a voice in this area.

On a local level, authoritative, society-based CPGs constructed in the manner described above can be used to inform and influence hospital guidelines to promote best practices. They also minimize the use of opinion-based guidelines and can be used to challenge payors’ decisions that are not based on high-quality evidence.

Finally, on a more personal level, evidence-based CPGs relieve the practicing physician of the burden of attempting to read and evaluate all the information being published in an area of practice. Evidence-based CPGs promote good clinical practice by reviewing, rating, and synthesizing this large amount of literature and then making an unbiased, evidence-based series of recommendations on clinical problems. In this manner, evidence-based CPGs serve to improve physician per­formance and patient outcomes.

William C. Watters III, MD, is chair of the AAOS Guideline and Technology Oversight Committee. His disclosure information is available online at www.aaos.org/disclosure

For more information, refer to the Institute of Medicine’s Clinical Practice Guidelines: Directions for a New Program, edited by Marilyn J. Field and Kathleen N. Lohr, available online at http://www.nap.edu/openbook