The word is in from the Yale Open Data Access (YODA) project: recombinant human bone morphogenetic protein (rhBMP-2), the product marketed as Infuse® by Medtronic, Inc., offers no appreciable benefit over autograft in spinal fusion surgery. The judgment is the result of an unprecedented move by the manufacturer, which turned over all patient data to the YODA project for review.

Two independent groups under the auspices of Yale University reviewed the complete patient data from clinical trials sponsored by Medtronic that were the basis of the U.S. Food and Drug Administration’s (FDA) approval of Infuse. Those trials became the subject of controversy in 2011 when the Senate Finance Committee expressed concerns that researchers with financial ties to the Minneapolis-based company failed to report adverse events in their published research, including sterility in men and harmful bone growth. The Spine Journal, the official journal of the North American Spine Society (NASS), then devoted an entire issue to a review of the available FDA data from the trials.

The journal’s editors were sharply critical of the studies reporting the research; editor-in-chief Eugene J. Carragee, MD, wrote that the studies underreported complications and overstated efficacy as compared with a conventional autograft procedure using iliac crest bone graft (ICBG).

In the wake of the controversy, Medtronic formed an arrangement with Yale University in which it released all of the study data and provided $2.5 million for the independent review. Yale enlisted groups from the Oregon Health & Science University (OHSU) and from the University of York in the United Kingdom to conduct the analysis.

YODA’s findings
Their reports, published in the June 18, 2013, issue of Annals of Internal Medicine, state that rhBMP-2 “provided little or no benefit compared with bone graft and may be associated with more harms, possibly including cancer.” Furthermore, the YODA groups found that the study authors underreported complications for both on-label and off-label indications and that they “selected analyses and results that favored rhBMP-2 over ICBG.”

The Oregon group said that it “found substantial evidence of reporting bias” in the studies as published and “no evidence that rhBMP-2 is more effective than ICBG.” It concluded that “on the basis of the currently available evidence, it is difficult to identify clear indications for rhBMP-2 in spinal fusion.”

The York group said that its analysis found that the published literature only partially reported the total outcome data known to have been collected. Although meta-analyses for main effectiveness (published data versus individual participant data) found that the “reporting of these results does not seem to be substantially biased,” adverse events were “incompletely and inadequately described in the study publications” and the way in which these events were presented was inconsistent, with the rationale for presenting some events and not others “rarely clear.”

According to Dr. Carragee, the YODA reports show that “the usual checks and balances of clinical research were seriously undermined.” He points out that the report authors made similar findings: “Early journal publications misrepresented the effectiveness and harms through selective reporting, duplicate publication, and underreporting.”

“We did not find a clear benefit over the traditional spinal fusion method,” she said. “Harms are common in both (rhBMP-2 and ICBG) groups, but except for cancer at 24 months, the increased risks for retrograde ejaculation, urine retention, and ectopic bone formation (with rhBMP-2) were not statistically significant.” This was true for both the one on-label use—single-level anterior lumbar interbody fusion (ALIF)—as well as for posterolateral approaches.

Although the detected complications were low in number and significance, “the sample size was small,” said Dr. Fu. “The studies were not designed to look for adverse events.” Her group’s report notes: “There has been no prospective, adequately powered study that specifically aimed to ascertain important harms by using adequate ascertainment methods.” It also states that her team “found no published trials truly independent of the manufacturer.”

Small studies problematic
Kevin J. Bozic, MD, MBA, of the University of California, San Francisco, and chair of the YODA Clinical Advisory Committee, agrees that small studies were problematic. “The revelation here was that in terms of the efficacy of rhBMP-2, what we found in the reported literature was consistent with what was in the independent review, but the adverse events were probably somewhat higher in the individual patient data than in the peer-reviewed publications.”

A primary reason for this finding related to small sample sizes, he said. “In any individual study, rates of adverse events may appear small, but when you aggregate them across studies, they become more apparent. The smaller the study, the more likely you are to miss something that could be clinically relevant.”

He agreed with Dr. Fu that another factor accounting for the low reporting of adverse events was imprecise categorization in advance.

“Complications were not well defined ahead of time,” said Dr. Bozic. “If you don’t define categories of severe, moderate, or low significance ahead of time, you may miss something. And if some complications were assumed, they wouldn’t be reported as adverse events. For instance, unexplained or persistent pain was considered an ‘expected’ outcome by certain investigators. So some investigators might not have reported the number of people who had pain. In some of the studies, however, the people in the BMP group reported more pain. If you don’t categorize persistent, unexplained pain as an adverse event ahead of time, you’d never know that.”

Indications for use
Although her group found no clear indication for using rhBMP-2, Dr. Fu noted that “we are not saying BMP is not an option. People make choices based on their preference.”

In their commentary, Dr. Bozic and Daniel Resnick, MD, MS, a neurosurgeon at the University of Wisconsin and a member of the YODA Clinical Advisory Committee, allow that rhBMP-2 may have some valid indications.

“In some procedures, such as ALIF,” they write, “graft harvest is a separate procedure and avoiding a second incision and associated graft site pain may be worth the exceedingly small risk for cancer.” On the other hand, they note, in posterolateral procedures, “locally harvested graft and ICBG are often available through the same incision. In these cases, it may not make sense to assume any increased risk, no matter how small.”

In cervical procedures, however, rhBMP-2 is associated with significant adverse events, wound complications, and dysphonia or dysphagia. In 2008, the FDA issued a safety warning regarding life-threatening complications in cervical applications, Drs. Bozic and Resnick write that rhBMP-2 should not be used at all in the cervical spine, absent a compelling reason, such as during a pseudarthrosis repair or other salvage procedure.

In any event, patients should be counseled on the relative harms and benefits and participate in the decision, they say.

“This device has a role in patients with certain risk factors, such as poor potential for healing and risk for pseudarthrosis,” said Dr. Bozic. “It would be unfortunate if the message delivered to clinicians was that we shouldn’t use rhBMP-2 because of a clinically insignificant risk for cancer.”

A new model?
The YODA project was the first time a manufacturer had released such comprehensive study data to outside reviewers. Dr. Fu said she hopes it won’t be the last time. “Unbiased, independent reviews of individual patient data would inform the public and physicians better than published industry-sponsored trials,” she said. “Hopefully, there will be more data sharing and reproducible research.”

Dr. Bozic, chair of the AAOS Council on Research and Quality and who served as the lone orthopaedic surgeon on the project team that launched and oversaw the YODA project, gave Medtronic credit for taking this step. “Very few companies would have subjected themselves to this level of scrutiny. They voluntarily released all the data to an ‘external auditor.’ They wanted to get at the truth.”

Medtronic did not respond to a request for comment, but in the Annals issue containing the reports, the company’s chief medical officer, Richard E. Kuntz, MD, MSc, wrote, “As shown by Medtronic’s initiation, support, and endorsement of this open-access approach with YODA, we intend to continue to advance our principles and leadership in pioneering new models of clinical research, data sharing, and transparency.”

Disclosure information: Dr. Carragee— Simpirica, Intrinsic Orthopaedics, Kaiser-NIH Grant, AO Foundation, Orthopaedic Research Education Foundation, The Spine Journal, NASS; Dr. Bozic—AAOS, American Association of Hip and Knee Surgeons, American Joint Replacement Registry, American Orthopaedic Association, California Joint Replacement Registry Project, California Orthopaedic Association, Orthopaedic Research and Education Foundation, YODA; Dr. Mick—NASS. Dr. Fu—YODA, Dr. Kuntz—no information provided.

Terry Stanton is the senior science writer for AAOS Now. He can be reached at tstanton@aaos.org

Bottom Line

• Under criticism for sponsored studies for Infuse rhBMP-2, Medtronic sponsored an independent data review.
• The YODA study groups found no appreciable clinical benefit to using rhBMP-2 instead of autograft in spinal fusion.
• Reported complications other than some pain and cancer were not statistically significant but were incompletely or vaguely reported in the original publications.
• Fuller financial disclosure and release of all data will improve the reliability of published studies.

What About the Money?
Sharp criticism had been leveled against the rhBMP-2 researchers who conducted the original studies, in part because they received significant compensation from the manufacturer. A report from the Senate Finance Committee noted that, over a 15-year period, Medtronic made a total of $210 million in payments to the physicians.

“We didn’t find any evidence that Medtronic was paying people to underreport,” said Dr. Bozic. “I don’t see this as a matter of people being dishonest. It was more a matter of individual investigators being unaware of the potential adverse consequences associated with rhBMP-2 due to small numbers of patients included in individual trials, lack of communication among investigators, and poorly defined categories of adverse events.”

Many involved in the project, however, called for greater transparency in research and publishing. “If a company is paying for the research, that should be disclosed,” said Dr. Bozic. “If investigators are being paid separately for other things, that should be made very clear, and it may not have been in this case. The readers should know that so we can come to our own conclusions about the design of the study and the reporting of the results. More specificity is always better. That will benefit all who are involved.”

Additional Information

• Read "Safety and Effectiveness of Recombinant Human Bone Morphogenetic Protein-2 for Spinal Fusion: A Meta-analysis of Individual-Participant Data…"
• Read "Effectiveness and Harms of Recombinant Human Bone Morphogenetic Protein-2 in Spine Fusion: A Systematic Review and Meta-analysis…"
• The Annals of Internal Medicine has published four editorals on this topic. Read more...
• Request the data and access additional information from YODA...