Orthopaedic surgeons took notice in late June when two members of the Senate Finance Committee sent a letter to Medtronic, Inc. (Minneapolis), maker of the recombinant human bone morphogenetic protein-2 (rhBMP-2) product Infuse. Sens. Max Baucus (D-Mont.) and Charles Grassley (R-Iowa) expressed concern about media reports that researchers conducting clinical trials on the product who had financial ties to Medtronic failed to report adverse events, including sterility in men and potentially harmful bone growth.
A week later, the orthopaedic community snapped to attention when the new issue of The Spine Journal came out. The entire edition was devoted to the topic of rhBMP-2 research and the impact of industry funding. Articles criticized what the editors called “corrupted research” and charged that researchers had allowed “unaccountable special interests to permeate medical research.”
The centerpiece of the special issue was a review article that examined 13 papers authored by physicians and researchers, some of whom had received millions of dollars from Medtronic. These studies reported essentially no adverse events or complications in the hundreds of patients in the 10 trials common to them. The Spine Journal editors, however, asserted that the studies underreported by a factor of 10 to 50 the complication rate found in Food and Drug Administration (FDA) data and other documentation.
The Spine Journal undertook the review of the 13 papers and associated data after readers—including some surgeons who participated in the trials—complained in letters about possible bias in articles on rhBMP-2.
The complications noted in the trial data include male sterility related to retrograde ejaculation, neuroinflammation, and osteolysis. In addition to underreporting these conditions, the editors say the sponsored studies may have handicapped control groups with unwarranted morbidity, long-term clinical failure, and exaggerated pain in patients undergoing fusion with autologous harvested iliac crest graft. The result, they say, was distorted results that favored the use of rhBMP-2.
“A challenge to integrity”
Eugene J. Carragee, MD, editor of The Spine Journal, served as lead author of both the review of the industry-sponsored trials and the editorial preceding it, “A Challenge to Integrity in Spine Publications: Years of Living Dangerously with the Promotion of Bone Growth Factors.” To do so, he recused himself as editor-in-chief for the rhBMP-2 issue, and the editor’s role was handled by Christopher M. Bono, MD.
According to the editorial, “After the original industry-sponsored trials for rhBMP-2, which were remarkable for the complete absence of reported rhBMP-2–related clinical adverse events, there came many reports of complications by authors unsponsored by the promoting company. With these reports, the safety profile of rhBMP-2 use has devolved, with troubling rapidity and little introspection about the processes, including idiosyncratic trial design, reporting bias, and peer review/publication shortfalls, that may have promoted widespread, poorly considered on- and off-label use, eventual life-threatening complications, and deaths.”
Dr. Carragee and the other authors quickly dismiss “the choirboy defense,” the protestation that orthopaedics is an honest profession and the integrity of researchers is unimpeachable. The argument that device royalties or consulting agreements with the sponsoring company would not affect a researcher’s objectivity, they say “fails to pass the test of minimum credibility.”
In an interview with AAOS Now, Dr. Carragee noted that the same 10 patient trials were the data source for the 13 reviewed papers. He also pointed out that much of the data for the trials are found in FDA documentation. “This confirms,” he said, “that adverse events and complications were omitted in the published papers.”
The burden of analysis
After the special issue was released, Thomas A. Zdeblick, MD, a prominent surgeon at the University of Wisconsin School of Medicine and Public Health, wrote to The Spine Journal, insisting that no attempt was made to hide data. Dr. Zdeblick, who has received substantial payments (royalties and other payments) from Medtronic over the years, was involved in the clinical trials and coauthored two papers on the product. He is also editor-in-chief of the Journal of Spinal Disorders & Techniques, which published two of the papers reviewed by The Spine Journal.
According to Dr. Zdeblick, the researchers tracked more than 40 types of adverse events during the trials, but because the incidence of these events was not statistically significant, they “were therefore not reported.” He also called the publication of the special issue and the commentaries alleging some type of cover-up, “naive and short-sighted at best and at worst inappropriate and irresponsible.”
This triggered another round of accusations, centered on the reporting of the incidence of adverse events (specifically retrograde ejaculation) among patients treated with rhBMP-2. According to Dr. Carragee and other editors of The Spine Journal, the original researchers failed to follow standard CONSORT (Consolidated Statement of Reporting Trials) recommendations for reporting complications in clinical trials, which require the reporting of “all harms or unintended consequences in each group.”
The CONSORT recommendations also note that reporting only the adverse events observed at a certain frequency or rate threshold and reporting only the adverse events that reach a statistically significant P value threshold in the comparison of the randomized arms are “common poor reporting practices for harms-related data.” Another standard notes that “statements about whether data were statistically significant without giving the exact counts of events” is also inappropriate reporting of data.
In discussing the situation with AAOS Now, Dr. Carragee said, “The authors of the papers appear to have reversed the burden of analysis. To prove that something works, you might need, say, a 95 percent or a 99 percent degree of confidence. But the authors of these studies applied that sort of statistical analysis to the safety data, which is inappropriate.
“Under the CONSORT recommendations and FDA rules, you don’t need to be 99 percent certain that something is causing cancer before you mention it,” continued Dr. Carragee. “The burden is the other way. If someone was 85 percent certain that a bridge was going to fall down if you drove over it, wouldn’t you want to know that beforehand? You wouldn’t say, ‘I only want to know if you are 99 percent certain.’”
AAOS Now requested a comment from Dr. Zdeblick for this article, but as this issue went to press, he had not responded.
In a statement to the press, the chief executive of Medtronic, Omar Ishrak, noted that although the articles in The Spine Journal “raise questions about researchers’ conclusions in their published peer-reviewed literature, the articles do not raise questions about the data Medtronic submitted to the FDA in the approval process or the information available to physicians today through the instructions-for-use brochure attached to each product sold.”
According to Dr. Carragee, one factor that led to publication of the flawed sponsored papers is that manuscript reviewers were provided only with highly selected summary data. “These days, with electronic capabilities, that’s not justifiable,” he told AAOS Now.
He also said that the current state of disclosure is “an absolute mess,” adding, “There are often no disclosures of conflicts. There are disclosures that are frankly inaccurate. Authors will say that a paper is not about a drug or a device when it is. The paper will say that the authors have no financial association with the manufacturer, when that’s clearly not true.
“What the journals are asking for is simply inadequate,” continued Dr. Carragee. “Some journals require disclosure of ‘more than $10,000,’ or ‘more than $30,000,’ between the author and the manufacturer. I think that’s a 1965 number. We are dealing with tens of millions of dollars now. You could burn through $30,000 just by getting some complimentary implants for bench testing.”
Dr. Carragee also told AAOS Now that he is troubled by issues of authorship. He noted that an article in The Wall Street Journal reported on a series of emails between authors and Medtronic inviting the company to make “changes/additions/deletions” to the manuscript. “It’s hard for us to know who is the primary writer of a paper,” Dr. Carragee said. “The impression is that the sponsor had some editorial control of the manuscript.”
The Spine Journal reveals financial information in its author disclosures, and under a newly implemented policy, disclosures will appear within the Methods section, not outside the article.
“Disclosures have to be as accurate as everything else in the paper, just like other variables about the study characteristics that we know affect the bias of outcome,” Dr. Carragee said. “We know that sponsorship of research is a global influence of outcomes. We have a history of what should be reported in the Methods and Results. By having it right there in the paper, the reader can see it and interpret it.” He also noted that the editor-in-chief and evidence and methods editors for The Spine Journal “have to be divested of almost everything.”
Despite the current controversy, Dr. Carragee does not believe that sponsored research or compensation to authors for related services is always linked to biased work or flawed reporting.
“If you have an interest and motivation and ability to do good clinical research, you should do it, and the company sponsoring you should pay the market rate,” he said. “But if at the end you are going to write advertising copy, we have a place in most of the journals for the advertising copy. If you are going to write a scientific paper, it has to meet scientific standards.”
Disclosure information: Dr. Carragee—Department of Defense, Department of the Army, Department of Justice, Intrinsic, Cytonics, Simpirica, Nuvasive (to institution), National Institutes of Health, AOSpine, Orthopaedic Research and Education Foundation, editorial boards of European Spine Journal, Spine, The Spine Journal, and The Journal of Spinal Disorders; Dr. Zdeblick—Medtronic Sofamor Danek, Anulex, Lippincott, Journal of Spinal Disorders, Cervical Spine Research Society, Lumbar Spine Research Society.
Terry Stanton is senior science writer for AAOS Now. He can be reached at firstname.lastname@example.org
JAAOS: When in doubt, disclose
As issues of disclosure and transparency assume greater importance, AAOS policies continue to evolve in the direction of more rigorous and fuller disclosure of conflicts by authors, editors, presenters, and others involved in the dissemination of knowledge, information, and instruction.
The Journal of the AAOS (JAAOS) has taken several steps over the years to make its disclosure program more robust. Disclosure information for all manuscript authors was originally collected on paper; today, all authors must complete the Academy’s online disclosure database and update it twice a year. In addition, JAAOS recently implemented an additional measure that checks an author’s name against public databases maintained by industry companies. Furthermore, JAAOS has joined 17 other major orthopaedic journals in adopting disclosure requirements outlined by the International Committee of Medical Journal Editors Uniform Requirements for Manuscripts Submitted to Biomedical Journals.
The extra steps that JAAOS takes to verify disclosures and conflicts, said Editor-in-Chief Jeffrey S. Fischgrund, MD, “involve a moderate amount of extra work, but are worth it because disclosure is so important. I think JAAOS and the Academy are ahead of the curve. We are proactive in ensuring we get as much disclosure as we can.”
JAAOS has also tightened its disclosure requirements for editors and reviewers. Dr. Fischgrund and the journal’s deputy editors complete the same “enhanced” disclosure that Academy board members do, reporting exact financial arrangements.
The review process for submitted manuscripts has built-in methods for avoiding bias, Dr. Fischgrund explained. If the possibility of bias exists—whether in the mind of a reviewer, deputy editor, or Dr. Fisch-grund himself—the paper is passed on to another qualified individual. “The system has a series of checks and balances,” he said.
Although determining bias can be a subjective process, Dr. Fischgrund said that JAAOS has a low threshold of doubt. “It’s not always black-and-white,” he admitted. “Reviewers have to gauge their internal barometers. Although it is possible for something to slip through the cracks, we rely on our reviewers’ internal decisions and we make it very easy to say no.”
When asked about The Spine Journal’s special issue, Dr. Fischgrund said, “It’s important for additional data to be published, even if the data are contrary to what has been previously published. Sometimes the reader just has to decide. Some of the more recent literature might be difficult to synthesize because it is in direct contrast to earlier reports. Anything that adds to the literature is valuable, but it has to be put in context with other studies. One paper by itself is not absolute.”
AAOS Now requires all authors to complete the Academy’s online disclosure database. Disclosure information for authors of scientific or clinical articles and for authors of studies covered by AAOS Now appears at the end of each article.
A second look at the data
In August, Medtronic, Inc. (Minneapolis), announced that it will provide $2.5 million to Yale University for an independent review of study data on its rhBMP-2 product. Yale will assemble a panel of experts and commission two academically recognized, publicly trusted clinical research organizations specializing in systematic reviews to conduct the analyses and ensure the integrity and reproducibility of the findings.
The company pledged to supply all available patient data from Medtronic-supported clinical trials, both published and unpublished, as well as all Food and Drug Administration adverse event reports. Medtronic will also post all of its clinical trials results information on ClinicalTrials.gov, and the company will retroactively register all premarket approval and postmarket clinical trials on the product that were completed before 2007, when such registration became mandatory. Finally, Medtronic said it is working with Yale to develop a program to provide researchers access to all data on rhBMP-2 by means of a defined registration process and website.
According to Rick Kuntz, MD, the company’s chief scientific officer, this “represents a novel and significant commitment to transparency and open-access scientific research.” Although Dr. Carragee agrees that this initiative represents “a big first step in the right direction,” he noted that “the Yale researchers will not be able to fix the study design flaws and assessment biases that were used in some of the original studies.”
The review study will be under the direction of Harlan M. Krumholz, MD, Harold H. Hines Jr. Professor of Internal Medicine, Epidemiology and Public Health at Yale School of Medicine. Dr. Krumholz will assemble a steering committee of 12 to 15 advisors, which will be chaired by Ezekiel Emanuel, MD, chair of the department of bioethics at the National Institutes of Health. The committee will include individuals with expertise in clinical issues, clinical reviews, clinical trial conduct, systematic reviews, and statistical analysis of clinical trial program data and ethical standards related to conflicts of interest, as well as a consumer representative.
“We understand questions have been recently raised about rhBMP-2 and look forward to sharing our conclusions publicly on the safety and effectiveness of this product at the end of our reviews of a full set of patient-level product data,” said Dr. Krumholz in a press release. “This project, including making the data accessible to all researchers, is intended to establish a landmark model for data transparency. If successful, we hope this new approach can become standard practice.”