HSR&D Home » Research » SDR 10-182 – HSR&D Study
Evaluation of VA Research Career Development Program
John W. Finney, PhD
VA Palo Alto Health Care System, Palo Alto, CA
Palo Alto, CA
Funding Period: July 2010 - January 2012
To ensure future generations of health services researchers working to improve health care for Veterans, HSR&D Service has supported a Career Development Award (CDA) Program since FY1991. However, the program had not been systematically evaluated.
This evaluation had two primary aims:
Aim 1: Compare the academic advancement and research productivity of HSR&D CDAs, NIH K awardees in health services research and AHRQ K awardees.
Aim 2: Predict the academic advancement, research productivity and VA retention and involvement of HSR&D CDAs.
Of 244 HSR&D CDAs from FY1991 through FY2010, 219 (90%) provided a curriculum vita (CV) and 118 (95%) of their 124 primary mentors sent a CV (they mentored 212 CDAs). Of 226 selected FY1991-FY2010 NIH K01/K08/K23 awardees in HSR comparable to that of CDAs, 154 (68%) provided a CV. Finally, 69 (76%) of 91 FY2000-FY2010 AHRQ K01/K08 awardees conducting VA-comparable HSR sent a CV. Productivity information from CVs was coded through 2010. The Hirsch h-index assessed publication citations.
We compared the full samples of HSR&D and NIH awardees, and the FY2000-FY2010 samples of HSR&D, NIH and AHRQ awardees, controlling for years since award. When multiple CDAs had been received, the mid-level RCD, MREP, or CDA-2 award was selected.
The three groups of awardees were between 37 and a little over 38 years of age, on average, when they received the awards. Men were more predominant among HSR&D CDAs relative to NIH awardees. Researchers from white-Hispanic or non-white groups were proportionately fewer among HSR&D CDAs than among NIH K awardees, with AHRQ K awardees having an intermediate percentage of minority group researchers. Almost two-thirds of all HSR&D and NIH awardees had an MD degree; the percentage for FY2000-FY2010 HSR&D CDAs dropped to 56%, whereas over 70% of AHRQ awardees had MD degrees.
Analyses of covariance for continuous outcomes and logistic regressions for dichotomous outcomes adjusted for pre-existing differences. No significant differences emerged in adjusted mean tenure track academic rank between the full samples of HSR&D and NIH awardees, or among the three groups of awardees from 2000 or later. Collectively, the 219 HSR&D CDAs had been the Principal Investigator (PI) on 1,147 grants of $5,000 or more (when funding amounts were provided; if not, the grant was still included), whereas the 154 NIH K awardees had been PI on 564 such grants. No significant differences emerged in the adjusted number of grants as PI emerged in any comparisons. However, HSR&D CDAs in the full sample who reported funding levels had been PI on significantly more grants of $100,000 than had the NIH K awardees in the full sample who reported grant amounts.
The 219 HSR&D CDAs had published 9,350 major journal articles (i.e., articles that were not editorials, letters to the editor, brief commentaries or book reviews), 2,480 as first or sole author. The 154 NIH K awardees had published 4,234 major journal articles, 1,497 as first or sole author. Relative to all NIH K awardees, HSR&D CDAs had a significantly higher adjusted mean number of major journal articles. However, no significant differences were found between the two samples in the adjusted number of major journal articles as first or sole author or in the h-index, or among awardees since 2000 on any of these indices. Publication trajectories over this time period were either increasing over time or relatively flat, with varying starting points. Finally, no significant differences emerged on the likelihood of having a journal editorship position or serving as a standing review committee member for a major granting entity, or on the number of postgraduate researchers subsequently mentored.
With years since the relevant award controlled, significant independent relationships of the predictors with seven outcomes were relatively few and generally modest in magnitude for HSR&D CDAs. Gender was unrelated to the outcomes, except that female CDAs tended to publish fewer major journal articles. Whereas minority group status was unrelated to any outcome, CDAs with MD degrees tended to have higher tenure-track academic positions and were more likely to have held at least one journal editorship position. CDAs with highest degrees from universities with higher (better) NIH research funding rankings were more likely to be involved in the HSR&D QUERI. Those with more years between receipt of a doctoral degree or, for MDs, completing a residency, and their CDA had higher tenure-track academic ranks, whereas having postdoctoral training was unrelated to any outcome. Although number of grants as PI prior to the CDA also was unrelated to outcomes, awardees who published more major journal articles prior to the award published more major journal articles subsequently and had higher h-indices.
None of the characteristics of primary mentors independently predicted CDAs' outcomes, except that mentors who had received more grants tended to advise CDAs who subsequently had lower citation indices. However, CDAs whose affiliated university had a better NIH research ranking subsequently published more journal articles and were more likely to attain a journal editorship position and to be retained in VA (80% of CDAs were in VA in 2010). Finally, the presence of larger HSR&D Centers was linked to receiving more grants as PI and greater involvement in the QUERI, but a lower likelihood of having had a journal editorship position. In analyses of data from 118 independent CDA-mentor dyads, many of the relationships between the predictors and HSR&D CDAs' outcomes were similar to those above.
We also compiled the accomplishments of the HSR&D Research Career Scientists in a separate report.
The results indicate that the HSR&D CDA Program is selecting promising applicants for CDAs and mentoring them effectively, as indicated by academic advancement and productivity which, at a minimum, are equal to the accomplishments of K awardees from the highly regarded NIH and AHRQ programs. Moreover, the HSR&D CDA Program has been successful in retaining researchers in the VA. This evaluation also provides findings that validate some criteria used by the HSR&D CDA Review committee in evaluating applications and suggest others upon which to focus.
External Links for this Project
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
MeSH Terms: none