How GTOC can help improve your practice

Last October, The New York Times published an op-ed piece by Billy Beane, general manager of the Oakland Athletics; retired Congressman Newt Gingrich; and Senator John Kerry, calling for a greater investment in evidence-based medicine (EBM) and pointing out that “studies have shown that most health care is not based on clinical studies of what works best and what does not—be it a test, treatment, drug or technology. Instead, most care is based on informed opinion, personal observation, or tradition.”

Not surprisingly, a sports analogy was used to underscore the point. Baseball teams that use sabermetrics, a statistics-based analysis method, to evaluate players and game strategies are not just winning more games, but are doing so with far lower payroll costs, noted the authors. In 2008, the New York Yankees, the Detroit Tigers, and the New York Mets—with a combined payroll of $486 million—failed to reach the World Series. Instead, it was the Tampa Bay Rays, with the second-lowest payroll in baseball, but with players and plays carefully chosen after analysis of stats like WHIP (walks and hits per inning). They also listed several examples of how “studying the stats” in health care has saved lives and reduced costs, but bemoaned the fact that this is the exception and not the rule.

The United States spends more than twice as much per capita on health care than most industrialized nations, yet has worse health quality outcomes. The op-ed piece urged healthcare policymakers to follow the lead of teams such as the Rays by developing policies that improve reimbursement (and liability protection) for physicians who follow clinical “best practices.” It concluded that the “best way to start improving quality and lowering costs is to study the stats.” But that’s easier said than done.

Surgeons have traditionally been like baseball managers who “built their teams and managed games based on their personal experiences and a handful of dubious statistics.” We have used data accrued from surgical case reviews based on physician choice and called them “science.”

Evolution of evidence-based medicine
Evidence-based medicine (EBM) and evidence-based clinical practice guidelines are relatively new to the quality improvement scene. The term only first appeared in the Journal of the American Medical Association in 1992.

EBM represents a sea change in the teaching and practice of medicine. Specifically, it de­emphasizes unsystematic clinical observations, pathophysiologic inference, and authority, and instead stresses “question formulation, search and retrieval of the best available evidence, and critical appraisal of the study methods to ascertain the validity of results.” For orthopaedic surgeons, this speaks to the critical importance of randomized clinical trials in the evaluation of surgical treatments.

Although randomized clinical trials are common in testing drugs, they are few and far between in surgery. Few surgeons are willing to let “chance” determine which surgery or hardware is used for their patients, and fewer still are willing to subject their patients to a “sham” procedure. Thus, creating the high-quality data on which to base decision making has been a challenge to all surgical subspecialties.

Early efforts at improving quality centered on the development of clinical practice guidelines, using a consensus-driven process. In the early 1990s, the Academy brought together thought leaders who reviewed and debated the current literature on selected topics and then developed a consensus report. Although these documents served as a helpful counterweight to payor-developed documents, which frequently seemed aimed at limiting care rather than improving quality, they were also problematic.

Ultimately nicknamed “BOGSATs” because they were created by a “bunch of old guys sitting around talking,” these “guidelines” were subject to the personal biases of the experts and were often felt to interpret the data based on the preferences and experience of the reviewing team. They were hard to defend and were generally honored in the breach.

But during the past 15 years, guideline development has grown up. Modern evidence-based guidelines are based on a systematic review of research that meets stringent quality standards and thus provide physicians with the best available practice standards. Evidence-based clinical practice guidelines (CPGs) provide a thorough review of the literature using a set of predetermined rules and, in doing so, minimize bias.

The AAOS, which is committed to the highest standards of patient care, established the Guidelines and Technology Oversight Committee (GTOC) in 2006 under the Council on Research, Quality Assessment, and Technology. The committee’s title and charges reflect the broad range and dynamic aspect of EBM.

GTOC is committed to developing practice guidelines for common clinical problems based on the best and most recent evidence available. But evidence-based clinical practice guidelines are not enough. AAOS members also need assessments of rapidly changing technology developments, particularly in situations where payors are reluctant to cover new products. The committee also helps determine which, if any, guidelines developed by other societies should be approved by the AAOS.

GTOC is making encouraging progress in all these areas. While continuing to focus on clinical practice guidelines, it is spearheading the development of a new type of document—the technology overview (TO). It has also initiated a process to ensure stringent review of guidelines developed by other professional groups.

New “stats”: CPGs and TOs
AAOS has developed four evidence-based CPGs. The Diagnosis of Carpal Tunnel Syndrome, The Treatment of Carpal Tunnel Syndrome, The Prevention of Symptomatic Pulmonary Embolism in Patients Undergoing Total Hip or Knee Arthroplasty, and Treatment of Osteoarthritis of the Knee are available on the AAOS Web site (www.aaos.org/guidelines) and will soon be published in the Journal of the AAOS. The GTOC is also involved in clinical practice guideline development on a wide variety of other topics and plans to collaborate with other medical societies in developing multispecialty guidelines.

TOs differ significantly from CPGs because they frequently do not contain the results of clinical trials or make recommendations about whether to use a device, drug, biologic, or procedure. TOs, however, can play a critical role in whether payors will pay for new technologies.

TOs and CPGs have a similar goal: To use evidence-based medicine to minimize bias, enhance transparency, and promote reproducibility. The assessment process uses specific rules of evidence that state what kinds of studies will or won’t be examined, searches for all available relevant studies, and documents all the relevant published findings.

TOs, in sum, are educational tools designed to assist readers in coming to their own conclusions about the available evidence. Thus far, AAOS has completed two TOs—Gender Total Knee Replacement and Locking Plates (see article on front page). A third overview is in production.

Learning from others
As a leader in the guidelines field, the AAOS is frequently approached by other societies seeking help with review, and sometime endorsement, of their guidelines. To maintain its standards of excellence, the Academy has developed stringent criteria covering both submission and standards. An outside guideline must be based on systemically reviewed evidence and be less than 5 years old. It must include a description of the funding source used for the development of the guideline as well as a statement guaranteeing the absence of any conflict of interest. Finally, the organization must submit a complete copy of the guidelines, including the supporting evidence table, to the AAOS for review.

The complete endorsement process can be found at www.aaos.org/Research/guidelines/Endorsementprocess.asp.

Putting EBM into practice
Despite the compelling rationale for evidence-based guidelines, development has been slow, and acceptance slower still. (See “What’s your excuse?” above.)

For years, fighter pilots believed they were the best judges of when to pull out of a dive when attacking a target. Only recently have they been convinced that modern aircrafts are too fast to permit the split-second judgments required. The American military is beginning to install automated equipment to sense when an aircraft is too close to the ground and override the pilot’s commands. What changed the pilots’ minds? Years of evidence that “controlled flight into terrain,” as it is politely phrased, accounted for the most aircraft accidents—and pilot deaths.

Although we as orthopaedic surgeons may be hesitant to turn our lives and practices over to “the men in green eyeshades,” the baseball and fighter pilot analogies, as well as the science, are clear: we can do a better job in almost everything we do, if we have better evidence.

Orthopaedic surgeons can indeed make a difference. We can promote utilization—starting with our own practices—and expedite dissemination to others. Are you willing to help?

Laura L. Tosi, MD, and Kenneth L. Moore, MD, are members of the Guidelines Oversight and Technology Committee. References for the studies cited in this article can be found in the online version, available at www.aaosnow.org

What’s your excuse?
Despite the compelling rationale for evidence-based guidelines, development has been slow, and acceptance slower still. The question is why? Here are six possible factors—and counterarguments from Drs. Tosi and Moore.

Evidence-based medicine (EBM) ignores clinical expertise.
On the contrary, EBM seeks to manage patients based on the best evidence available. Although it may seem that EBM requires physicians to give up some autonomy in decision making, in fact, EBM is based on a wide range of clinical experiences.

EBM requires randomized, controlled trials.
Although randomized, controlled trials are preferred, they are not the only source of evidence. EBM looks at a variety of sources.

EBM is about numbers, not patients.
EBM starts with patients in general, and the clinical practice guideline focuses on the patient at hand.

EBM can’t be applied to all the needs of the individual patient.
Using science to guide our decisions does not exclude weighing the values and perceptions of the individual patient. Perhaps the best definition of EBM is as follows: The practice of evidence-based medicine requires three equally important ingredients—best evidence plus physician experience plus patient preference.

Clinicians can’t keep up with the evidence behind EBM.
Keeping up with the evidence may be hard, but it’s not impossible. For example, the National Center for Biotechnology Information allows anyone interested to register for “my NCBI” to help identify high-quality research.

We don’t believe EBM is really evidence-based.
Yes, it is; disbelievers are urged to volunteer for the Guidelines and Technology Oversight Committee so they can learn firsthand just how stringent the evidence requirements are for each AAOS clinical practice guideline and technology overview.

References:

• Beane B, Gingrich N, Kerry J. How to take American health care from worst to first. New York Times. Op-Ed October 24, 2008. Available at http://www.nytimes.com/2008/10/24/opinion/24beane.html?ref=opinion
• Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 2008;300(5):1814-6.
• White B. Making Evidence-based medicine doable in everyday practice. Family Practice Management. 2004(Feb): 191-8.
• Poolman RW, Petrisor BA, Marti RK, Kerkhoffs GM, Zlowodzki M, Bhandari M. Misconceptions about practicing evidence-based orthopedic surgery. Acta Orthopaedica. 2007;78(1):2-11.
• Sackett DL. Clinical epidemiology, what who and whither. J Clin Epidemiol. 2002;55(12):1161-6.