We will be performing site maintenance on our learning platform at learn.aaos.org on Sunday, February 5th from 12 AM to 5 AM EST. We apologize for the inconvenience.

AAOS Now

Published 8/1/2019
|
Daniel A. London

Resident Research and Fellowship Applications: Quantity or Quality?

Orthopaedic surgery residents learn to balance many priorities during training, one of which is participation in research. A common dilemma residents struggle with is whether to participate in fewer but higher-quality research projects, which likely require greater time commitments, or in a greater number of (but possibly lower-quality) research projects. This tension takes on added significance for residents applying for fellowship training positions as they try to decipher whether the quantity or quality of their research has greater influence on their likelihood for success in the fellowship match.

The first question to consider is whether research experience is even necessary to be successful in applying to fellowship programs. The answer depends on a resident’s fellowship and career goals. If a resident is interested in a fellowship and a career focused on private practice, then the importance of research in the application process is less significant. For residents interested in academics, however, research experience is essential. Research as an activity is universally viewed as a positive and important component of residents’ fellowship applications, and some fellowship programs use publications as a way to assess residents’ true interest in pursuing an academic career. Furthermore, some fellowship programs (most likely those with an academic focus) have ranked research involvement as the most important criterion when considering fellowship applicants.

Other studies have demonstrated that most fellowship programs across subspecialties ranked research as one of the lowest considered criteria when they created their applicant match lists. Regardless of how research experience is used as a selection criterion, an increased number of publications has been shown to be predictive of a future academic career, which may be related to the implication that a resident who takes on multiple projects and guides them to publication has a “get-it-done” attitude that some academic fellowship programs focused on productivity may find appealing. Clearly, quantity of research can make an impression, but there is also value in a singular, high-quality project that may take longer to execute but results in a greater impact on orthopaedics, its body of knowledge, its practices, and patient outcomes.

If quality of research matters, the next question to consider is how to best assess research quality. Although there are many metrics, there is no set standard. Quality could be defined based on the type of research products being created (level of evidence) or the impact factor of the journals that publish the work. No one would argue that a well-done prospective, randomized, controlled trial is higher-quality research than a retrospective case series, or similarly that publication in a high-impact, peer-reviewed journal is superior to a publication in a predatory, pay-to-publish online access journal. The significance of an individual author’s contribution to a study is difficult to assess but may be revealed in author order. Finally, study impact or relevance can be evaluated by metrics such as the number of citations individual manuscripts or authors have—the i10-index, which measures the number of publications with 10 or more citations—or the h-index, which is based on a calculation involving the number of publications and the number of times each is cited.

Unfortunately, these metrics are challenging to apply to differentiate residents’ research experience. The relatively short course of residency, coupled with the time it takes to initiate and complete a project, create a manuscript, navigate the peer-review publishing process, and have other authors cite that work, limits these varying metrics’ value in accurately representing the quality of a resident’s research portfolio. Furthermore, if such quality characteristics are not clearly stated on an application, it may be unlikely that fellowship directors thoroughly evaluate applicants’ research quality given the volume of applications. Assessing the quantity of resident work also can be difficult; given the short time frame of residency, multiple projects may be in various stages of completion at any given time and, as such, are not necessarily reflected in citations.

The challenge of assessing the importance of research quantity and quality is not unique to fellowship applicants; the same question arises for academic orthopaedic surgeons in their quest to be promoted. In such situations, the number of publications and the quality of the research, as well as surgeons’ overall academic and clinical productivity, matter. Ideally, fellowship directors examining fellowship applications give equal weight to the quality and quantity of research. However, with an increasing number of applicants, expediency of quantifying research productivity may seem advantageous. This can be accomplished by tallying research products, including peer-reviewed articles of primary research, review articles, case reports, book chapters, grants, and poster and podium presentations—easily accomplished by a quick review of a CV.

Despite the variety of tracking mechanisms for both research quantity and quality, a clear answer is lacking on whether either or both of these characteristics matter in the fellowship application process. What is known is that orthopaedic residents applying to fellowship training are being more academically productive in regard to the number of research products created with each passing year.

In the eyes of fellowship programs, the importance of such productivity varies. Although research is important, status as first author has been shown to not be nearly as critical. The overall message from the literature appears to be that research involvement is important, but that specific measures of quality may be difficult to decipher, with perhaps a slight emphasis on quantity metrics. This perception is echoed by Bradford Parsons, MD, fellowship director of the Mount Sinai Shoulder Surgery Fellowship: “When it comes to research, it is definitely something we look for in applicants. Quality can be hard to measure, but increased productivity can be a differentiator between applicants with similar profiles.”

The implication is that orthopaedic residents applying to fellowship programs should not be focused on their research quality or quantity solely for their fellowship applications. Rather, they should keep their final career goals in mind and pursue projects that reflect those goals, more than an arbitrary number of publications. On a higher level relating to the advancement of our profession, quality in research should be the focus over quantity. In that regard, there is room for improvement. As such, residents should focus on the quality of research over quantity. Residency provides plenty of stressors; residents’ volume of research efforts does not have to be one of them.

Daniel A. London is a post-graduate year-5 orthopaedic surgery resident at the Icahn School of Medicine at Mount Sinai in New York City. He will be undertaking hand training in Cincinnati at the Mary S. Stern Hand Surgery Fellowship.

References:

  1. Bastian S, Ippolito JA, Lopez SA, et al: The use of the h-index in academic orthopaedic surgery. J Bone Joint Surg Am 2017;99:e14.
  2. Baweja R, Kraeutler MJ, Mulcahey MK, et al: Determining the most important factors involved in ranking orthopaedic sports medicine fellowship applicants. Orthop J Sports Med 2017;5:2325967117736726.
  3. DeFroda SF, Shah KN, Safdar O, et al: Trends in research productivity of residents applying for orthopedic sports medicine fellowship. Phys Sportsmed 2018;46:61-5.
  4. Egro FM, Vangala SK, Nguyen VT, et al: Hand surgery fellowship selection criteria: a national fellowship director survey. Arch Plast Surg 2017;44:428-33.
  5. Grabowski G, Walker JW: Orthopaedic fellowship selection criteria: a survey of fellowship directors. J Bone Joint Surg Am 2013;95:e154.
  6. Namdari S, Jani S, Baldwin K, et al: What is the relationship between number of publications during orthopaedic residency and selection of an academic career? J Bone Joint Surg Am 2013;95:e45