ORS Clinical Research Forum examines “wrong turns”
That the Clinical Research Forum sponsored by the Orthopaedic Research Society (ORS) and devoted to “avoiding wrong turns” in the development of new products and therapies would include the issue of recombinant human bone morphogenetic protein-2 (rhBMP-2) is no surprise. In 2011, rhBMP-2 came into a less-than-flattering spotlight over questions not only about complications associated with its use—much of it off-label—but also about how the original industry-sponsored trials could have reported essentially no adverse events.
According to the man most responsible for bringing that question to the fore, Eugene J. Carragee, MD, of Stanford University, the published findings by researchers exhibited the “warning flag of warning flags: when it is too good to be true.”
He pointed to the eight large industry-sponsored trials, involving about 600 patients treated with “an enormous dose” of rhBMP-2, without any reports of complications. “If that’s true,” Dr. Carragee said, “then rhBMP-2 would be 20 times safer than a short course of ibuprofen and penicillin.”
Calling research into question
In an issue of The Spine Journal devoted to rhBMP-2 questions, Dr. Carragee’s editorial expressed his view that “the safety profile has devolved, with troubling rapidity” because of process flaws. These shortfalls, including poor trial design and reporting bias in peer review/publications, “may have promoted widespread, poorly considered on- and off-label use, eventual life-threatening complications, and deaths.”
In his view, the trials conducted by investigators who received large sums of money from the manufacturer failed to address or accurately report complications that had been detected in previous research. These complications included neuroinflammatory reactions of retrograde ejaculation and radiculitis, infection and wound problems, and cancer promotion or carcinogenicity. The studies continued to depict impressively positive results even though, Dr. Carragee said, multiple reports of complications arose as soon as BMP came into general use in 2004.
He noted that stories about BMP were on the front page of major newspapers. “That’s too late for warning signs,” he said. “That’s a head-in-the-sand sign.”
The root of the problem with the research, he said, might have been the premise of the studies: that for single-segment fusion, the use of ileac crest bone graft (ICBG) “is so morbid that it had to be avoided.” Yet none of the data supported that premise, he said. “The early data, through 3 months, did not show improvement, in multiple randomized clinical trials. If the back pain from ICBG harvesting was so bad, you should at least see it at 6 weeks.”
Dr. Carragee related the experience and actual role of The Spine Journal, which published some of the studies at issue. He said the journal had amassed a backlog of 39 articles on BMP complications. He noted that on his return from active military duty, he found the journal’s editorial process “was being widely criticized.”
Some of the complaints were from surgeons who were involved in the study. They claimed that the published data were inaccurate and that multiple complications had not made it into the article. Lack of full disclosure, poor reporting of adverse events, and data that didn’t agree with documents submitted to the U.S. Food and Drug Administration (FDA) were among the problems.
Identifying the problems
In a systematic review of available data from the FDA and other sources, the editors found more problems, including the following:
- Even in the larger trials, FDA data indicated complications with rhBMP-2 “may be common and serious,” but went unreported.
- The presence and the magnitude of the authors’ conflicts were “astounding,” involving millions of dollars. The disclosures were often “unintelligible or inconsistent.”
- The pathology used to assess the ICBG harvesting was poor and seemed biased against the control group.
- The control group methods and technique, as selected for both posterior approach methods were potentially handicapped by design bias against the controls.
Among the lessons learned, Dr. Carragee said, were to beware of original safety concerns that are not systematically evaluated in a study and to beware of confusing a drug trial with a device trial. He also said to watch for adverse event reporting “that looks primarily at a statistical confidence in ‘proving harm’ rather than statistical confidence in safety. This is a bait-and-switch in which you talk about how safe something is but you report the statistics as harm.”
Surgeons should avoid extrapolating massive off-label product use from extremely limited safety data, should be suspicious if original data sets or summaries were not available to reviewers or readers, and should be wary of vague disclosures of funding and potential conflicts of interest.
Major concerns that exist before a study should be systematically reported on in the trials. For example, comparative rates for retrograde ejaculation, a concern associated with an anterior approach, were not reported in any of the five papers based on the FDA trials, even though the data showed five times the rate of the complication in the BMP group versus the control.
Similarly, the fact that concern about malignancy was not mentioned in the articles “should have been a big warning sign.”
Dr. Carragee noted that drug trials look at acute events versus cumulative occurrences. “Even large adverse event effects are going to be missed if you are going to add up everything over 5 years.” This problem was detected in reporting on urinary retention, delayed infection, and back and leg pain.
Working off the label
Thomas Mroz, MD, of the Cleveland Clinic, addressed some general issues associated with off-label—also called “physician-directed”—use.
BMP, he noted, was discovered in 1965 by Marshall Urist, MD, but it was not until 2002 that one form—rhBMP2—was approved by the FDA for use, and only in one-level anterior lumbar interbody fusion with an LT-Cage (Medtronic). Its use in any other procedure is off-label. FDA regulations and other applicable laws recognize that physicians may prescribe or administer any legally marketed product for an off-label use, according to the physician’s well-informed medical judgment for the best interest of the patient.
Asking rhetorically, “If it works, what’s the problem,” Dr. Mroz cited the numerous problems previously detailed, noting that many of them occurred in off-label usage and with high-dose usage.
Extrapolating from sales data, Dr. Mroz estimated that about 165,000 patients received rhBMP-2 in 2011, and 148,000 will do so this year. He estimated that two thirds of this usage is off-label. A recent survey of SpineLine’s readership, for example, found that most of the responding surgeons use rhBMP-2 for various off-label fusion procedures.
The common way to obtain data about the safety and efficacy of the product is to review the literature, but Dr. Mroz noted that such a review will be based on a relatively small number of patients versus the large number who actually receive the product. Another consideration is the considerable heterogeneity in study design, reporting, execution, and the conflict-of-interest
Registries and insurance databases have great potential for following large numbers of patients, Dr. Mroz said. Such databases would also enable indications to be monitored and outcome comparisons to be made across regions and over time. However, such database management is complex and expensive.
Presenting an overview of the evaluation of industry reviewed trials in the context of rhBMP-2, Kevin J. Bozic, MD, MBA, first noted that merits of the industry’s involvement include its providing a source of capital, infrastructure, and risk for innovation.
“Physician-innovators, who tend to be risk-averse by nature, benefit from close collaboration with industry, which provides the capital, infrastructure, and willingness to take risk that are necessary for this type of innovation,” he said. Dr. Bozic alos noted that the device industry provides much-needed support for clinical trials.
Industry selection of investigators can lead to biased interpretation of results, Dr. Bozic said, noting that surgeons who receive consulting and royalty fees often have financial conflicts that may influence their objectivity in interpreting the results of a clinical trial. This issue can affect those who read manuscripts describing the results of industry-funded clinical trials, and the opinions of “thought leaders” often carry out-sized influence with clinicians.
The public perception of bias is also an issue involved in industry funding, whether or not there is actual bias in a study. In terms of sharing results, Dr. Bozic noted that industry-funded research is proprietary, with no requirement for publication or dissemination. Many clinical trials that are conducted are never published; even among published trials, not all data are revealed.
Dr. Bozic chairs the Clinical Steering Committee of the Yale Open Data Access project (YODA), which is seeking to address some of the concerns with industry sponsorship of clinical research. Funded by Medtronic, Inc., which manufactures the rhBMP-2 product used in the studies, the project aims to “increase transparency and enhance the public trust in industry-funded clinical trials by facilitating the independent assessment and dissemination of data relevant to the benefits and harms of drugs and devices.”
Panelist Dan M. Spengler, MD, of Vanderbilt University, asked: “Can we accept new products for human use based solely upon studies that have been supported by the producers of the products? Do lead institutions and investigators need to be non-conflicted?” Responding to his own question, he said, “Answers will need to consider financial and personnel resources as well as time delays to market.”
Jeffrey C. Wang, MD, of UCLA, observed that “biologics will play a key role in the future of othopaedic surgery. Scientific advances are made by scientists. We must critically review the evidence and make decisions based on good
Disclosure information: Dr. Carragee—Intrinsic Therapeutics, Simpirica; Dr. Mroz—Pearl Diver, AO Spine; Dr. Bozic—Agency for Healthcare Research and Quality, National Institutes of Health, Robert Wood Johnson Foundation, California HealthCare Foundation, Center for Health Quality and Innovation, Blue Cross Blue Shield Association, Integrated Healthcare Association, Pacific Business Group on Health, Ingenix, InVivo Link; Dr. Spengler—Journal of Bone and Joint Surgery–American, Musculoskeletal Transplant Foundation; Dr. Wang—Bone Biologics, Axiomed, Amedica, Corespine, Expanding Ortho, Pioneer, Axis, Syndicom, VG Innovations, Pearldiver, Flexuspine, Fziomed, Benvenue, Promethean, Nexgen, Electrocore, Surgitech, Medtronics, Biomet, Stryker, Seaspine, Aesculap, Osprey, Alphatech.
Terry Stanton is the senior science writer for AAOS Now; he can be reached at firstname.lastname@example.org
March 2012 Issue
Search AAOS Now
- AAOS Now
- Current Issue
- Editorial Information
- Writers' Guidelines
- News in 10
- The Annual Meeting Daily Edition of the AAOS NOW
S. Terry Canale, MD
E-mail the Editor
Volume 7, Number 12
- Cover Story
- Clinical News & Views
- Research & Quality
- Managing Your Practice
- Your AAOS