We will be performing site maintenance on our learning platform at learn.aaos.org on Sunday, February 5th from 12 AM to 5 AM EST. We apologize for the inconvenience.

AAOS Now

Published 10/1/2014
|
Eric F. Swart, MD; Eric C. Makhni, MD, MBA; Samir Mehta, MD

Cost-effectiveness Research: How to Implement Study Findings into Clinical Practice

As healthcare costs and expenses continue to rise, interest in cost-effective medicine increases. This has led to an expansion in cost-effectiveness analyses (CEA) to evaluate the economic impact of various common orthopaedic interventions. CEA, however, is not the same as cost-benefit analysis for treatment. The results from CEA may not always be straightforward, and incorporating these results into clinical practice may be quite challenging.

This article serves as a brief primer on the terms, techniques, and interpretation of CEA, using a recently published article (Prevention and Screening Programs for Anterior Cruciate Ligament [ACL] Injuries in Young Athletes: A Cost-effectiveness Analysis. J Bone Joint Surg Am, 2014 May 7) to illustrate salient points.

Goal of CEA
The primary goal of CEA is to evaluate the results of different treatment or decision options by comparing the financial impact (costs) to the clinical outcomes (effectiveness) of each choice. Costs are almost always measured in monetary terms (eg, dollars). However, they can also be measured in terms of the expenses incurred by a patient or a healthcare provider/institution, or the overall cost to society in general—the broadest and most commonly used perspective.

Effectiveness can be measured in terms of specific outcome measures (eg, dollars per ACL injury prevented or dollars per death avoided), in which case the study is considered a cost-benefit analysis. However, these results are less generalizable, making comparisons across multiple studies difficult. In these cases, the preferred method is a cost-utility analysis that expresses the clinical outcome measure in quality adjusted life years (QALY), a more general measure of a health state with respect to a theoretical ‘perfect’ health state.

Although these studies are broadly comparable to other studies, measuring health utility is a relatively subjective quantity that is very difficult to measure. The use of QALYs has also been criticized as being too variable and not applicable globally.

It is also important to clarify what treatment options are being evaluated. Sometimes a given treatment is compared to the historically known progression of a disease left untreated (eg, surgical treatment of a rotator cuff tear compared to therapy and observation). In the ACL CEA article, multiple different strategies to prevent ACL injuries were compared, including universal screening for ACL injury risk with a prevention protocol for high-risk athletes, implementing a universal prevention protocol, and no screening or prevention protocol. This study would be considered a standard cost-effectiveness study, because QALY is the primary unit of measure for intervention effectiveness, although results are also given in injuries prevented.

Once the treatment options are defined, a model is created to calculate the effects of the decisions, ultimately leading to different results with different total costs and measured health outcomes. In the ACL article, clinical branch points (eg, positive or negative result of the screening test) were used to create a tree of possibilities (Fig. 1). The authors then worked forward through the different choices to determine the ultimate net outcomes. This is called rollback or decision tree analysis.

Results from the analysis are given in terms of which decision performs better, usually in terms of dollars spent per QALY gained. Historically, $50,000/QALY has been used as a general cutoff for defining an intervention as “cost effective,” although recently it has been suggested that $100,000 or higher per QALY may be a more reasonable goal. This study showed that universal training not only had better outcomes (increased QALY due to fewer ACL injuries), but also saved money because fewer surgeries were required. In this case, that decision is said to dominate the other choices.

Interpreting the findings
After reviewing any cost-effectiveness study, the main question for the practicing orthopaedic surgeon becomes, “Now what?” The reader must interpret the findings of the study to determine if, and how, these findings can be incorporated into practical practice management. The relevance of a CEA is a function of the quality of the inputs—and outputs—used in the study, the range of outcomes in the sensitivity analysis, and the strength of the conclusions that are made. These factors ultimately combine to determine how applicable the results are to a given surgeon’s practice.

The quality and relevance of inputs used in the study are important initial considerations. Readers must determine whether the value of these inputs relate to those found in their local markets and clinical population. For example, the ACL case study population included athletes 14 to 22 years old who were involved in organized sports with a relatively high incidence of ACL injuries. If the same intervention is applied to a different population (older, less formal sports involvement), the inputs may be different and the results may not apply directly.

In addition to scrutinizing the values of the inputs used in the study, readers must also study how the conclusions vary according to inputs in the sensitivity analysis. The sensitivity analysis involves varying the inputs through a range of possibilities and examining the effects on the model’s conclusions. If the results of the study indicate that the conclusions are very sensitive to small changes in a single input variable, readers must understand that these conclusions are limited in scope.

For example, if the cost-effectiveness of an implant is relatively insensitive to its unit price, readers need not be as concerned about the impact of changing price and can focus on other inputs used in the study’s model. The ACL article reports results that are relatively insensitive to the major input variables, meaning the conclusions are likely valid over a wide variety of possible scenarios.

After studying the inputs used in the study—along with the impact of these inputs on the overall conclusion—readers must then assimilate the study as a whole and determine if the presented conclusions are strong or weak. The ACL example incorporates data from numerous prospective, randomized trials as well as high-level meta-analysis, so readers can have relative confidence in the findings. If the study had relied largely on lower evidence level studies, and the inputs were based on a variety of assumptions without clear supporting data, the conclusions would be less reliable.

Interpreting CEA studies can be challenging, because they use analytic techniques and terms that many orthopaedic surgeons do not routinely use. However, they can provide valuable insight into ways that practicing surgeons can help ensure that the services they provide add value and are not simply the newest or most expensive. Balancing concern for cost-effectiveness with an individual patient’s clinical situation enables orthopaedic surgeons to deliver high-quality patient care while simultaneously containing excess costs and improving efficiency.

Eric F. Swart, MD, is an orthopaedic resident at Columbia University Medical Center in New York; Eric C. Makhni, MD, MBA, is one of the 2013–2014 AAOS Washington Health Policy Fellows; Samir Mehta, MD, a 2003–2004 AAOS Washington Health Policy Fellow, is now chief of the division of orthopaedic trauma and assistant professor of orthopaedic surgery at the Hospital of the University of Pennsylvania.

Bottom Line

  • Cost effectiveness analyses (CEA) evaluate the results of different treatment or decision options by comparing the financial impact (costs) to the clinical outcomes (effectiveness) of each choice.
  • Results from the analysis are given in terms of which decision performs better, usually in terms of dollars spent per quality of life years (QALY) gained; historically, $50,000/QALY has been the general minimum standard.
  • Interpreting and implementing CEA results into an orthopaedic practice may be challenging, but can help ensure that the services provided actually add value.

References:

  1. Swart E, Redler L, Fabricant PD, Mandelbaum BR, Ahmad CS, Wang YC: Prevention and screening programs for anterior cruciate ligament injuries in young athletes: A cost-effectiveness analysis. J Bone Joint Surg Am 2014;96(9):705-711. doi: 10.2106/JBJS.M.00560
  2. Ranawat AS, Nunley R, Bozic K: Executive summary:  Value-based purchasing and technology assessment in orthopaedics. Clin Orthop Relat Res 2009;467(10): 2556-2560.  doi: 10.1007/s11999-009-0908-5. Epub 2009 Jun 4.
  3. Gold MR, Siegel JE, Russell LB, Weinstein MC: Cost-effectiveness in Health and Medicine. New York, Oxford University Press, 1996.
  4. Rutten F: Economic evaluation and health care decision-making. Health Policy 1996;36(3):215-229.
  5. Andrawis JP, Chenok KE, Bozic KJ: Health policy implications of outcomes measurement in orthopaedics. Clin Orthop Relat Res 2013;471(11):3475-3481.
  6. Novak EJ, Vail TP, Bozic KJ: Advances in orthopaedic outcomes research. J Surg Orthop Adv 2008;17(3):200-203.
  7. Mather RC 3rd, Koenig L, Acevedo D, et al: The societal and economic value of rotator cuff repair. J Bone Joint Surg Am 2013;95(22):1993-2000.
  8. Braithwaite RS, Meltzer DO, King JT Jr, Leslie D, Roberts MS: What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care 2008;46(4):349-356.
  9. Pierce RG, Bozic KJ, Hall BL, Breivis J: Health care technology assessment: Implications for modern medical practice. Part II. Decision making on technology adoption. Am J Orthop (Belle Mead NJ) 2007;36(2):71-76.