ORS Clinical Research Forum tackles methodologic issues in RCTs

Orthopaedic clinical trials are “challenged,” according to Mohit Bhandari, MD, one of the organizers of the Orthopaedic Research Society’s (ORS) Clinical Research Forum. The challenges, as outlined during the forum, include both design and participation issues.

Despite the fact that orthopaedics is a device-driven medical specialty, only 84 of 1,000 industry-sponsored medical device trials registered in clinicaltrials.gov between October 2005 and June 2009 were orthopaedic trials. Most orthopaedic devices are approved through the 510(k) process, which enables approval of a device found to be equivalent to a previously approved device without the need for clinical trials.

Orthopaedic clinical studies get ‘C+’
Although randomized controlled trials (RCTs) are conducted in orthopaedics, their quantity and quality are lacking, according to Dr. Bhandari. “Many ‘C+’ RCT studies are being published, fueling a relative explosion of meta-analyses (Fig. 1),” he said.

“Because the quality of the initial trials is problematic, the results from a pooled analysis—even a meta-analysis with a thousand patients—will not provide a definitive answer,” he continued.

Clinical trials in orthopaedics face numerous pitfalls. For example, in a surgical trial, patients are randomized to receive treatment A or treatment B. This design, however, is based on several assumptions—about the surgeons’ skills, the patients’ willingness to accept the assigned treatment, and the number of participants needed to ensure valid data.

“We have to assume that participating surgeons can perform treatment A or treatment B with equal efficiency, ability, and expertise—and this is a fundamental challenge of surgical trials,” explained Dr. Bhandari.

To demonstrate how this differential expertise may influence an outcome, Dr. Bhandari reviewed a multicenter hip fracture trial conducted in the United Kingdom. The trial weighed hip replacement against surgical fixation in treating displaced intracapsular hip fractures in otherwise healthy individuals age 60 or older.

“The consultant-surgeons were twice as likely to perform the operation when the patient was randomized to hip replacement,” he explained. “But when the patient was randomized to fixation, the procedure was typically left to the junior-most staff.”

The results found a clear advantage for arthroplasty compared to fixation, resulting in policy changes in the United Kingdom. “More importantly, it had major effects on clinical practice,” Dr. Bhandari said.

Critical elements for successful RCTs
Blinding and power are two other critical elements, according to Dr. Bhandari. An investigator should not be able to determine the treatment allocated for the next patient in the trial. “Concealment is a very simple and fundamental concept but may be more misunderstood than any other concept in orthopaedic trials,” he explained.

Instead of using envelopes that can be “bright-lighted” to show the next treatment allocation, remote telephone- or Internet-based systems for randomization should be used to ensure concealment.

When blinding is not used, the estimates of benefits tend to be inflated. Although surgeons can’t be blinded in orthopaedic clinical trials, others can and should be. “In examining 72 RCTs reported on in The Journal of Bone and Joint Surgery from 1988 through 2000, we found that at least two thirds of the surgical trials did not blind outcome assessors, patients, or data analysts,” said Dr. Bhandari.

“The size of the trial is always the elephant in the room. How many patients do we have to recruit for the data to be valid?” he continued. Although a power of 0.80 is often used, he suggested the power should be set to 0.90 or higher to ensure that a difference is found if one exists.

If a study is not appropriately powered, a false negative finding is possible. A trial with too small a sample size may not detect a difference when one is present. “We may inadvertently disregard important therapies,” he said.

Studies with small sample sizes do “a great deal of data dredging and subgroup analyses; as a result, they sometimes get a positive finding, but this could be a false positive,” explained Dr. Bhandari.

“In the United States, and at times in the world, we are making clinical decisions based on studies with an average sample size of fewer than 100 patients,” he said.

Preferences complicate RCTs
Even when enough patients are recruited for a clinical study, additional challenges remain—particularly in large pediatric trials.

The Bracing in Adolescent Idiopathic Scoliosis Trial (BrAIST) involves 27 clinical sites in two countries with three healthcare systems—United States, Canada, and the Shiners Hospital system.

“Bracing has been the standard, nonsurgical treatment for scoliosis,” said Stuart L. Weinstein, MD, primary investigator. “However, it has never been subjected to a rigorous evaluation of either its efficacy or its effectiveness.”

BrAIST—a Phase III, multicenter, prospective, randomized, and partially blinded clinical trial—is designed to determine whether the use of a thoracolumbo-sacral orthosis for a curve of less than 50 degrees lowers the risk of progression relative to watchful waiting alone.

The trial required 384 patients to detect a 50 percent reduction in surgical rates (assuming a surgical threshold of a 50 degree curve), according to Dr. Weinstein. “We came to this endpoint by surveying patients across the United States. The patients and their parents actually determined what type of effect size they wanted to see to make it worthwhile to participate in the trial.

“This is the first scoliosis study to have randomization, objective brace/dose monitoring, standard radiographic protocols and measurement, comprehensive radiographic clinical and psychosocial testing, and diversity of the participating sites,” he explained. This is also the first clinical trial in which patients determined their desired outcome.

The planning grant provided for a study of patients and parents from five diverse centers across the United States and Canada to determine a reasonable percentage of randomization. When individually approached, 55 percent of parents and 45 percent of children would participate in a hypothetical trial. But when parent and child responses were matched, the participation rate dropped to 30 percent. As a result, the investigators adopted a proposed randomization rate of 25 percent.

When recruiting for the trial
actually began, however, the randomization rate was just 19 percent. As a result, the investigators changed the design to a partially randomized, preference trial. This enabled patients “to be randomized, to go to the preference arm and choose their treatment, or to opt not to participate,” Dr. Weinstein explained. The BrAIST trial currently has 152 patients in the randomized arm and 226 in the preference arm.

“Preference is a very important aspect in this trial,” Dr. Weinstein said. “Both parents and children have very strong treatment preferences. In pediatric orthopaedics, a surgeon will invariably get the question, ‘What would you do if this was your child?’ It’s very hard for some of the investigators not to influence the parent’s decision with their own treatment bias.”

Forum tackles issues with orthopaedic trials
The ORS “Clinical Research Forum: Learning from the Past, Looking to the Future” was held on Jan. 15, 2011, at the ORS Annual Meeting in Long Beach, Calif. The forum was organized by Theodore Miclau, MD; Kristy L. Weber, MD; George F. Muschler, MD; and Mohit Bhandari, MD.

The day-long seminar was divided into the following four sessions: “Setting the Stage,” “Lessons Learned,” “Methodological Issues in Clinical Trials,” and “Advanced Concepts in Clinical Trials.”

Annie Hayashi is the development and communications manager for the Orthopaedic Research Society. She can be reached at hayashi@ors.org

References:

1. Keating JF, Grant A, Masson M, Scott NW, Forbes JF: Displaced intracapsular hip fractures in fit, older people: A randomized comparison of reduction and fixation, bipolar hemiarthroplasty and total hip arthroplasty. Health Technol Assess 2005;(9)41:1-65.
2. Bhandari M, Richards RR, Sprague S, Schemitsch EH: The quality of reporting of randomized trials in The Journal of Bone and Joint Surgery from 1988 through 2000. J Bone Joint Surg Am 200284-A(3):388-396.
3. Poolman RW, Struijs PA, Krips R, et al: Reporting of outcomes in orthopaedic randomized trials: Does blinding of outcome assessors matter? J Bone Joint Surg Am 2007;89(3):550-558.