A study found low levels of agreement between orthopaedic surgeons and radiologists on the radiographic assessment of osteoarthritis (OA) of the knee, including factors such as disease characteristics and recommended treatment. Justin Magnuson, MD, research fellow at the Rothman Orthopaedic Institute at Thomas Jefferson University in Philadelphia, will present the study on Thursday, March 24, during the AAOS 2022 Annual Meeting.

In an interview with AAOS Now, Dr. Magnuson and his coauthors, Arjun Saxena, MD, MBA, FAAOS, and Andrew M. Star, MD, FAAOS, both also from the Rothman Orthopaedic Institute, explained why their team chose to investigate this topic. “While systems to classify OA have been described and studied, objective scales are rarely utilized in common practice,” they said. “As a result, physicians, including radiologists and orthopaedic surgeons, often utilize the subjective terms ‘mild, moderate, or severe’ OA.”

The goal of this study was to evaluate agreement between orthopaedic surgeons, musculoskeletal (MSK) radiologists, and general radiologists on the assessment of radiographs of patients with knee OA. They compared surgeons’ and radiologists’ ratings of OA severity and location and whether they recommended total knee arthroplasty (TKA).

The study used 105 deidentified radiographs from patients who presented to a high-volume arthroplasty center. No patient history was included with the radiographs. Each image was reviewed independently by two fellowship-trained arthroplasty surgeons, two general radiologists, and one MSK radiologist. Classifications for OA severity were “mild,” “moderate,” or “severe,” and locations were described as medial, lateral, patellofemoral, or a combination thereof.

The researchers used Kappa-Fleiss tests for agreement among the overall cohort, surgeons only and radiologists only, all surgeons versus all radiologists, and between surgeons and general or MSK radiologists specifically. Kappa-Fleiss values of 0 to 0.3 were considered “no true agreement,” between 0.3 and 0.5 as “weak agreement,” and higher than 0.5 as “moderate agreement.” Moderate agreement or higher was considered reliable.

Overall, there was “low agreement both between and within specialties in the evaluation of OA severity and location,” Dr. Magnuson said. “Surgeons and MSK radiologists had moderate agreement in whether they would recommend surgery, while there was lower agreement with general radiologists.”

Across all comparison groups, there was weak agreement on OA severity (all ≤0.41) and location (all <0.20). in the cohort of all reviewers, there was weak or no true agreement on all parameters. there also was weak agreement between surgeons and radiologists on severity, patellofemoral location, and surgical recommendation, with no true agreement on medial, lateral, or tricompartmental location. overall, there was no or weak agreement on identification of medial and lateral compartment oa and no agreement regarding tricompartmental oa.>

There were overall low levels of agreement between orthopaedic surgeons and the two radiology specialties specifically (Fig. 1). Regardless of specialty, there was no agreement on location and weak agreement on severity. Notably, there was moderate agreement between orthopaedic surgeons and MSK radiologists regarding recommendation for TKA, compared with weak agreement on that parameter between surgeons and general radiologists.

The team found disagreement on radiographic interpretation even between surgeons for OA severity (0.46) and identification of lateral OA (0.30). “We were surprised to find a low level of agreement regarding location and severity even within physicians of the same specialty,” the authors commented. However, there was moderate (0.50) agreement between surgeons on the recommendation for TKA.

Radiologists also demonstrated low intraspecialty agreement. Between MSK and general radiologists, there was moderate agreement on patellofemoral OA (0.51); however, there was weak agreement on OA severity and no agreement among radiologists on the remaining parameters.

“These findings suggest that interpretation of radiographic findings in patients with OA is highly subjective, indicating the need for further standardization of classifications which are easily implemented into clinical practice,” the study authors concluded. “These results also highlight the importance of patient history in surgical decision-making.”

From a clinical standpoint, a “lack of a common language to describe arthritis can lead to difficulty communicating with third parties, such as payers,” commented Dr. Magnuson and colleagues. “Lack of standardization makes objective evaluation of treatment alternatives difficult. To study knee-replacement outcomes, it is important to account for this uncontrolled variable.”

Regarding the study’s limitations, the authors commented that the orthopaedic surgeon reviewers were “two high-volume arthroplasty surgeons [and that] all of the readers practice in a metropolitan area, so these results may not be generalizable to all orthopaedic surgeons and areas.” However, Dr. Magnuson commented, “The low level of agreement among all parties is very suggestive and motivates us to design further studies to continue to evaluate this problem.”

Based on these findings, the authors suggest, “Future studies should seek to develop reliable and reproducible methods for interpreting radiographs that show strong agreement both within and between specialties.”

The study will be on display as Poster P0597 on Friday in Academy Hall, from 7 a.m. to 5 p.m.

Drs. Magnuson, Saxena, and Star’s coauthors of “Agreement between Orthopaedic Surgeons and Radiologists in the Interpretation of Radiographs of Knees with Osteoarthritis” are Francis Sirch, BS, and Raja Nicholas Kyriakos, MD.

Rebecca Araujo is the associate editor of AAOS Now. She can be reached at raraujo@aaos.org.