Published 5/1/2015
Brett Walker, DO; Harvey Smith, MD; Kristen E. Radcliff, MD

Risk Stratification, Outcome Assessment, and Online Transparency

What do they mean for orthopaedic patients?

Publicly available outcomes data have fostered an increased perception of transparency with respect to results of procedures and patient experiences with healthcare providers and hospital staff. Surgical outcomes measures are inherently affected by the procedure’s technical complexity and the patient’s medical situation. Accordingly, interest in risk characterization and stratification is increasing.

An entire industry has evolved to provide consumers with aggregated outcome data on both providers and hospitals, as well as subjective characterizations of patients’ perceptions of the care episode. Surgical outcome and quality data have become important commodities with financial implications for health reporting companies, providers, and patients. This article will review currently available risk stratification algorithms and examine outcomes assessments and their ties to reimbursement, as well as online resources for patients.

Risk stratification systems
Discussing surgical risk is a critical component of any preoperative consultation. Direct risks related to surgical invasiveness and indirect risks related to the patient’s overall health must be considered. As outcomes reporting and reimbursement changes occur, orthopaedic surgeons must be increasingly diligent to appropriately stratify patient risk to optimize results and provide quality care.

The oldest risk stratification algorithm is the American Society of Anesthesiologists (ASA) physical status classification system. The ASA score has been shown to be an important predictor of perioperative complications following arthroplasty and trauma. However, some critics have suggested that the system is too broad, noting that it does not account for patient age or type of surgery.

Another commonly used risk stratification algorithm is the Charlson Comorbidity Index (CCI). The CCI includes patient medical diagnoses and was originally developed to predict 1-year mortality in nontrauma hospital patients. The CCI may also have some additional value in postoperative risk stratification.

A third system was developed by the American College of Surgeons. The purpose of the National Surgical Quality Improvement Program (ACS-NSQIP, http://riskcalculator.facs.org) is to stratify a patient’s individual risk and to track intraoperative and postoperative complications. The NSQIP includes additional patient-specific parameters (such as age, sex, functional status, comorbidities, body mass index, and ASA score) along with procedure-specific details. Although the system provides more outcomes information, it requires the use of an online calculator, which may limit its practicality.

Currently no risk stratification scheme is universally accepted. Different algorithms can result in different estimates of postoperative complication rates for the same procedure. Additionally, retrospective analyses and administrative reviews may underestimate the rates of complications compared to prospective studies.

At The Thomas Jefferson University system, engaging the medical, anesthesia, nursing, and surgical team members resulted in implementation of specific risk stratification algorithms and successful reduction of the most frequent complications (pulmonary, renal, delirium, and cardiac).

Current algorithms are not specific to orthopaedic surgery, not universally accurate, inherently biased because they are based on unreliable administrative datasets, and are largely unproven in prospective studies. As orthopaedic surgeons, we need additional research and guidelines about best practices for risk stratification to provide appropriate benchmarks and enable the public, payers, and patients to judge our work fairly.

Outcomes measurement
Outcomes measurement is a critical component of modern orthopaedic surgery practice. Physicians, through the scientific literature, may read about patient-reported outcomes measures, with at least 2-year follow-up. In contrast, patients are exposed to quality metrics based only on process measures analyzed in a proprietary fashion with short-term follow-up.

Several third-party companies, including USNews, Consumer Reports, and Leapfrog have created “grading” websites for physicians and hospitals. According to these companies, the grades are based on publicly available data from the Centers for Medicare & Medicaid Services (CMS), the Center for Disease Control and Prevention, and the American Hospital Association. However, each company uses different proprietary measurement algorithms to create their respective rankings. Many even include highly subjective patient or physician surveys when determining rank.

These complex algorithms have not been subject to rigorous peer review to determine their validity. Deeper scrutiny of individual website procedures reveals a lack of consistency in methodology among companies. Depending on the primary outcome measured, the weight allocated to each section, and the results of surveys, rankings can vary significantly. For example, depending on the structure of the outcome measures, some of the most well-respected academic hospitals in the country receive relatively poor scores.

As medically complex patients are increasingly managed at tertiary care facilities, reporting algorithms that prioritize length of stay and medical complications may bias against facilities that provide care to a disproportionate percentage of higher-risk patients. These variations may lead to patient confusion about the quality of care provided at medical centers. Interested stakeholders—including patients, providers, and payers—should collaborate to define appropriate, uniform quality metrics for online professional reporting. As more information becomes available on the Internet, healthcare ranking websites must be subjected to increased scrutiny by the medical community.

Recently, CMS published an unprecedented amount of data on surgical trends and quality, as part of its shift to link hospital and physician compensation to recorded outcomes, including the possibility of payment penalties based on adverse outcomes or readmissions data. As a consequence of this model, hospitals and surgeons who care for a high-risk medical population or perform high-risk surgical procedures may be at risk for poor report cards and financial penalties.

The most common complications following orthopaedic surgery are often medical in nature and not directly related to surgical technique. Perioperative medical optimization and patient selection based upon medical history are important trends in orthopaedic care. This raises the concern that high-risk patients may eventually have difficulty finding care as hospitals and physicians seek to maintain appropriate reimbursement and public scores.

As the number of physicians directly employed by hospitals increases, physicians and hospitals are uniquely aligned to participate in careful patient selection that may screen out high-risk patients. Such avoidance behavior has already been described in neurosurgical literature due to malpractice concerns. Additionally, the practice of complication avoidance may ultimately limit patient choices as emphasis shifts to potentially less effective, lower morbidity procedures. In some cases, this approach may not improve general and disease-specific outcomes as much as more complex procedures.

Validated patient-reported outcomes measures are time-consuming to acquire, frequently have a proprietary nature and cost, and generally are collected longitudinally over a period of 2 years or more. Perioperative event data such as 30-day readmission, return to operating room, medical complications (such as anemia, pulmonary complications, or renal failure), length of stay, discharge disposition (home versus rehab or skilled nursing facility) are readily available for abstraction from episode of care billing data. Consequently, many hospital and physician ranking systems rely on episode of care data that may not ultimately correlate with longer term (1- or 2-year) validated patient reported outcome measures.

One way to decrease perioperative complications and shorten length of stay is to focus on healthier patients and lower complexity procedures. Accordingly, the emphasis on perioperative rubrics of questionable long-term significance may have the unintended consequence of limiting access to care for those with the highest disease burden and those who require more complex procedures.

Cautious consideration of what outcomes measures are relevant and how to accurately reflect patient and procedure complexity in these grading systems is crucial. It is incumbent upon professional societies to take a leadership role and actively engage with consumer reporting industries.

Patient resources
Finally, patient experience reporting venues enable individual users to review and comment on hospitals and physicians. This platform raises many specific questions concerning anonymity, legality, and the potential for fraud. Users are able to rate and leave purely subjective comments concerning their experiences and interactions with physicians and hospitals. Websites may have no way to confirm that the individual submitting a ‘review’ was indeed a patient, nor any mechanisms to prevent an individual from submitting multiple reviews of a single episode of care.

The anonymous nature of these postings is problematic for several reasons. First, anonymity makes any regulation or monitoring extremely difficult. Secondly, statements may not be well clarified and could be taken out of context. Finally, anonymous postings do not provide a frame of reference for the physician to adequately address negative reviews.

Recently, online reviews have been the topic of much legal debate. In the private sector, many lawsuits have been filed against online reviewers, accusing several of libel. Court rulings have mostly been in favor of commenters, categorizing most posts as “opinion,” which is protected by the first amendment. Under the Communications Decency Act of 1996, internet service providers or websites are generally immune to libel lawsuits based on anonymous posts.

Successful libel litigation requires that the review contain an unsubstantiated statement of fact. For example, suggesting that a physician was negligent or that a hospital had no license could be considered defamation if untrue.

Although businesses have the freedom to openly respond to reviewers, physicians are bound by privacy laws such as HIPAA. This makes meaningful replies all but impossible. Of course, patients are entitled to comment on positive and negative experiences; however, improved regulation and transparency of this burgeoning field may help to eliminate false and fraudulent reviews.

The recent trend toward public transparency and reporting of outcomes has had both positive and negative effects. Due to public pressure, orthopaedic surgeons, medical providers, and hospitals have partnered to study and reduce perioperative complications at their centers by developing risk stratification algorithms. Despite this, public reporting forums may still have the unintended consequence of creating barriers to care in high-risk patients who may benefit from complex interventions.

Additionally, an online quality reporting industry has flourished that reports process-measure–based outcomes that could result in confusing patients. As medicine moves forward in the era of outcome measurements and reporting, it is important to anticipate and avoid unintended consequences that may cause inadvertent harm.

Ideally, professional societies, such as AAOS, as public education and advocacy organizations, may be the ideal vehicles to educate the public and payers about best practices in risk stratification and quality in orthopedic care.

Brett Walker, DO, is a member of the department of orthopedic surgery at Michigan State University, McLaren Oakland Hospital; Harvey E. Smith, MD, is assistant professor of orthopaedic surgery at the Hospital of the University of Pennsylvania and the Veteran’s Administration Medical Center; and Kristen E. Radcliff, MD, is at the Rothman Institute, Thomas Jefferson University, department of orthopedic surgery.