Authors say use of patient satisfaction scores may be misguided
A study evaluating the correlation between patient-reported outcomes and outpatient Press Ganey Scores in patients undergoing total joint arthroplasty (TJA) found little if any correlation between in-clinic satisfaction as expressed in Press Ganey Medical Practice Survey scores and improvement in function, mental health, and pain.
The study's authors say the results, presented at the 2017 annual meeting of the American Association of Hip and Knee Surgeons by senior author Jeremy Gililland, MD, of the University of Utah, cast doubt on the utility of Press Ganey scores as surrogate measures of healthcare quality, especially when reimbursements become tied to these metrics.
Jessica M. Kohring, MD, a University of Utah resident, led the study.
Dr. Gililland noted that he has firsthand experience with the use of Press Ganey results at his own institution, which "publishes our Press Ganey results online as a method of demonstrating transparency." The chief medical officer at the University of Utah previously stated that the reported scores "give visitors a powerful tool to make informed decisions about our physicians and providers."
"The implication is that Press Ganey scores in some way are a measure or metric of physician quality," Dr. Gililland continued. He noted his own physician webpage displays his Press Ganey scores, which for 5 of 8 categories addressing the patient's view of the quality of his care are 4.8 on a 5-point scale, with two 4.7s and one 4.6, along with a 3.7 for clinic wait time. Although these scores are uniformly positive, the patient comments paint a less conclusive portrait.
"One comment said I am super, I listen to all my patients' concerns and their questions," Dr. Gililland said. "Another one says that I don't listen and I tend to be argumentative with my patients. One patient reports that I am candid and spend time with them, while another says I'm in a hurry and I only do what I think is best for them—which I thought is my job."The question is," he continued, "What do these comments and scores mean, and are they in any way, shape, or form a measure of quality of the physician? With this study, we sought to determine if that was the case. Our goal was to see if Press Ganey outpatient scores were really correlated with total joint arthroplasty outcome scores."
Thus, the study compared Press Ganey scores from patients who underwent TJA with patient-reported outcome (PRO) scores—specifically those of the Patient Reported Outcomes Measurement Information System (PROMIS) physical function computer adaptive test (PF CAT), along with the PROMIS global health assessment. The purpose was to determine if the overall Press Ganey score correlated with patient-reported outcome scores in patients undergoing primary knee or hip replacement.
Surveying the survey
The Press Ganey Medical Practice Survey consists of 24 questions divided into the following six subdomains: access, moving through the patient's visit, the nurse or assistant, the care provider, personal issues, and overall assessment of the practice. Each question measures responses on a Likert scale ranging from 1 (indicating very poor) to 5 (indicating very good). Responses are converted to a 0- to 100-point scale, and the mean overall score for all answered questions within an individual subdomain is used to calculate the score for that subdomain.
Specifically, the investigators sought answers to four questions:
- Is there a correlation between a random selection of Press Ganey scores and PRO scores from the same visit?
- Is there a correlation between preoperative Press Ganey scores and PRO scores from the same visit?
- Is there a correlation between postoperative Press Ganey scores and PRO scores at one year or greater after TJA from the same visit?
- Is there a correlation between Press Ganey scores and the magnitude of change in PRO scores at a minimum 1-year follow up?
They retrospectively reviewed secondary data from outpatient clinical encounters at a single center, the University of Utah. They screened all patients who underwent a primary total knee or hip replacement from January 2014 to February 2017, but excluded patients who had undergone multiple joint replacements and those without Press Ganey scores.
The authors assessed results for 540 patients (Fig 1.). The average patient age was 64 years (range 28–95) with a mean body mass index of 29 kg/m2 (range 16–50). Fifty-six percent (n = 304) of patients were female and 56 percent (n = 300) were total knee arthroplasty patients; the remaining 44 percent (n = 240) were total hip arthroplasty patients.
In a random selection of patient visits (n = 540), there was little, if any, correlation between the Press Ganey score and all PRO scores including the PF CAT, the Global 10 mental health, Global 10 physical health, and the Global 10 NPS. For the preoperative visits (n = 266), the correlation between Press Ganey score and all PRO scores "was also negligible at best," the authors note. Similarly, there was no significant correlation between postoperative Press Ganey score and the PRO scores.
"We also wanted to see whether these patients actually got better with their surgery," Dr. Gililland said. "If we look at our postoperative outcome scores, we can see that on average, they improved, in all measures, to a level consistent with the mean scores of the U.S. general population."
A misleading measure?
This study, the authors state, "contributes to the growing body of literature in both the nonorthopaedic and orthopaedic patient populations that patient satisfaction scores may not be a true measure of the quality of health care provided. It appears that nonmodifiable patient characteristics and variables outside the control of the healthcare provider significantly impact patient satisfaction with care."
The authors conclude: "As providers, we should be aware of these scores and strive to serve our patients better and improve their satisfaction. However, in the current environment of legislated improvement of quality of care, patient satisfaction with their healthcare experience has been confused with healthcare quality. This study demonstrates that there is little, if any, correlation between a patient's in-clinic satisfaction (Press Ganey scores) and their improvements in physical function or mental health, or decreases in pain in a population of primary TJA patients. While patient satisfaction may be important to providers or healthcare systems wishing to increase volume and referrals, it is unlikely that efforts focused on improving satisfaction scores are likely to lead to improved outcomes."
Dr. Gililland commented, "While Press Ganey scores can be important, as we want to know if our patients are satisfied with the care we are giving, I don't think that an effort to improve Press Ganey scores is going to improve outcomes for our patients. I think that our findings question the utility of these scores as surrogate measures for healthcare quality."
Addressing the use of Press Ganey results at his own institution, Dr. Gililland said, "When we started doing this our scores were being sent to us monthly. They came with a percentile rank, which could be frustrating. Your score would be, say 4.73 one month and it would be 4.81 the next month, and your percentile would be, say, 25th percentile the first month and next month 99th percentile, just with a simple change. It's frustrating when you are told one month you are in the 35th percentile and next month you're doing great. It can seem like a random noise generator."
Dr. Gililland's coauthors are Jessica M. Kohring, MD; Christopher E. Pelt, MD; Mike B. Anderson, MSc; and Christopher L. Peters, MD.
Terry Stanton is the senior science writer for AAOS Now. He can be reached at firstname.lastname@example.org.
- Patient satisfaction scores such as those from Press Ganey patient surveys are increasingly used to measure and publicly rate physician quality.
- This study compared results from Press Ganey surveys and functional outcomes measured via PRO instruments in patients undergoing total joint arthroplasty.
- Results showed that there is little, if any, correlation between a patient's in-clinic satisfaction (Press Ganey scores) and their improvements in physical function or mental health, or decreases in pain.
- It appears that nonmodifiable patient characteristics and variables outside the control of the healthcare provider significantly impacted patient satisfaction with care.
- The authors concluded that efforts focused on improving satisfaction scores are not likely to lead to improved outcomes.