My first pass evaluation of the K99-R00 program

Jul 17 2014 Published by under Uncategorized

In 2006, NIH initiated a new grant program The Pathway to Independence award program. This transition award was a direct response to a National Academy of Sciences report "Bridges to Independence" that highlighted the long time between degree completion and the initiation of an independent research career. This program using a combination of the K99 and R00 mechanisms (and are often referred to a "Kangaroo" awards).

The first awards were made in FY2007. In 2011, Drugmonkey posted a qualitative evaluation of the K99-R00 program and noted that some K99-R00 recipients had gone on to receive R01 funding. However, it was too early to perform a more quantitative analysis. However, now sufficient time has passed that quantitative data are more interpretable.

I analyzed the grant trajectories of the 182 K99 recipients awarded in FY2007.

Here are the key results:

170 (or 93%) transitioned to the R00 phase, indicating that they had obtained an appropriate faculty position.

Of these 170 individuals, 101 (or 59%) have obtained at least 1 R01 grant through the present. In addition, 4 had obtained New Innovators (DP2s), 1 had obtained a Pioneer (DP1) and 1 had moved to the NIH intramural program.

The timing for obtaining R01 grants is shown below:

2007 plot

The plot shows that R00 awardees received R01 support as early as FY2009. The number of R00 awardees receiving their first R01 peaked in FY2012 and has been falling. The line to FY2014 is dotted as this fiscal year is not yet complete. The cumulative percentages of R00 awardees with R01 support are as follows:

FY2009-4%; FY2010-14%; FY2011-27%; FY2012-43%, FY2013-53%; FY2014-59%

 

For comparison, I also analyzed the 178 K99 awardees from FY2008.

158 (or 89%) transitioned to the R00 phase.

Of these 158 individuals, 66 (or 42%) have obtained a least 1 R01 grant through the present. In addition, 6 had obtained New Innovators (DP2s) and 1 had moved to the NIH intramural program.

The timing for obtaining R01 grants is shown below:

2007 2008 plot

The FY2008 cohort is obtaining R01 grants with a similar but slightly longer time delay compared to the FY2007 cohort, due at least in part to the impact of the ARRA funds on the FY2007 cohort. For the FY2008 cohort, the determination of the overall fraction obtaining R01 funding and the final trend will depend on the results through the end of FY2014 and, perhaps, beyond.

The cumulative percentages are:

FY2009-0.6%; FY2010-1.2%; FY2011-7%; FY2012-16%, FY2013-37%; FY2014-42%

From these data, there can be no doubt that the the K99-R00 program has been effective in transitioning individuals from postdocs to faculty positions. It is clear that individuals with these awards compete well for such positions through some combination of the skills and qualities that allowed them to successfully compete for the K99 award and the interests of the hiring institutions in having faculty with some extant funding and a track record of obtaining such funding. The role of this program in influencing hiring has been discussed in another Drugmonkey post.

The results in obtaining R01s is also quite impressive to me. Of course, it is not clear what percentage of these individuals would have obtained positions without the "kangaroo hop". One comparison could be to look at F32 recipients who went on to faculty positions during the similar period, but this will take some work. Again, the factors that have led to this success include scientific skills, grant writing skills, relatively good contacts with NIH staff, funds for preliminary data, and others.

It is less clear how much impact this program has had on the timing of the transition to independence if this is defined by obtaining independent R01 funding. There is still a substantial lag between obtaining a faculty position and obtaining R01 funding, comparable to what Drugmonkey posted with my data from NIGMS (although these results were from times where success rates were somewhat higher). Nonetheless, overall it is hard not to see the K99-R0o program as a substantial success.

 

21 responses so far

  • chemicalbilology says:

    This is really interesting!

    It does look like first R01 awards peak at about 5 years post-K99/R00 award start year. If there are about 2 years of additional postdoc/mentored training, then that corresponds to about year 3 or 4 of the faculty position. Timing-wise, I wonder if that's really any different than any other new TT person? That seems to be about when everybody needs to secure their first R01 in order to continue on in this job, and also corresponds to about when any typical study section culture would see the new PI as having had enough independent activity and preliminary data to get scored well enough. But it would be interesting to look at whether the % of awardees getting R01s in that timeframe is higher than in the general TT population--however, I don't even know whether those data are available in any high-throughput format...i.e. year of first R01 awarded linked to year PI started position...

  • qaz says:

    This is the wrong question. The K99/R00 identifies the top postdocs in a given year. The question is whether these postdocs would be doing any differently if the K99/R00 didn't exist. In my anecdata, my observations are that these postdocs were getting TT jobs and R01s at similar timescales and rates that you are reporting here.

    In my opinion, the K99/R00 solved a problem that didn't exist and created new ones. The K99 is very unfair to people who stumbled at any point in their careers because it has a very tight and strict timelimit. It has become a marker for "top postdoc" that has unfortunate additional criteria. Because many universities have made the K99 an unwritten (sometimes written!) criteria for hiring, they have effectively outsourced their hiring decisions to NIH.

    I think getting rid of the K99 would be a very good thing.

    • datahound says:

      As I noted in the introduction, the K99-R00 program was created to address a real (and persistent) problem, namely the long time to independence for young scientists. There was a specific call for such a program from a good National Academy of Sciences report. Whether the program has achieved this goal is difficult to access, at least with publicly available data, but it has succeeded in some ways.

      With regard to universities "outsourcing their hiring decisions to NIH", in my opinion, this is the fault of the universities that are doing this, not the fault of the NIH program. Many outstanding young scientists have not received K99-R00 awards and any university that is limiting themselves to the K99-R00 pool is taking a very short-term and limited view. At the same time, the K99-R00 program does provide a relatively open pathway for young scientists to improve their chances for a short at a successful academic career.

      • qaz says:

        It is not clear at all that the K99 has reduced the time to independence for the top caliber of scientists. Your data does not support that claim. In the few years before the K99 appeared, top postdocs were getting faculty jobs after 3-4 years of postdoc. Yes, many postdocs were also getting faculty jobs after long postdocs. Now those second set of postdocs are completely screwed.

        If the K99/R00 program has reduced the average time to faculty, it has done so by removing the students who stumble, not by reducing the time people spend in postdocs.

        Can you give me one concrete example (comparing equivalent postdocs from before to K99 winners) in which the K99/R00 program has "succeeded"?

        I definitely agree that the fault of universities outsourcing their hiring decisions to NIH is the fault of the universities. But the government rules change the incentive structure of the universities' decisions, and IMHO the K99 produces counter-productive incentives.

        • datahound says:

          qaz: I understand your concern and agree in some ways but disagree in others. The available data supporting the idea the K99 program reduced the time to independence are not strong and more analysis of this point needs to be done by NIH. Thus, I mostly agree on this point.

          You also ask me to give an example of someone who has succeeded specifically because of the program. A major challenges of programs like this is the selection bias that goes into them, that is, would the individuals who get the awards have succeeded anyway (without any help from the award)? Unfortunately, I do not have access to the data that would help with such an analysis (although NIH does). For example, one could look at individuals who received good K99 scores but were not funded and follow their career trajectories. Anecdotally, I know of particular individuals who where not clear superstars as graduate students or postdocs, but who were creative, wrote a strong K99 proposal that got funded and have done well subsequently. Can I prove that they would not have had the same trajectory without the K99? Of course not, but it seems to have helped and the program did provide an open path to try to improve one's chances.

          With regard to those who have done (or are currently doing) longer postdocs, I do not think they are "completely screwed." Unfortunately, no good data are available, but I certainly know of individuals who have obtained reasonable tenure track positions with long postdocs, even recently.

          Overall, the current situation is quite challenging. The NIH budget has been flat for more than a decade and many institutions are contracting due to that and other forces such as decreased state support, constraints on clinical revenue at academic medical centers, and the overall state of the economy. This makes it tough for individuals trying to find positions. But, would it have been better if individuals who receive K99-R00 awards d0 not transition into faculty positions and them earn R01s and other funding subsequently?

          • qaz says:

            Yes, of course, "completely screwed" is an overstatement. But the evidence is very strong that the K99 is being used as a marker for "good prospect" (probably accurately so).

            Certainly, I'm glad that the K99/R00 people are having success. The question I have is (a) would the money be better spent elsewhere, and (b) are the restrictions on the K99 appropriate?

            Is there any evidence that universities increased their positions available because of the existence of the K99s? (I doubt it.)

          • DrugMonkey says:

            Is the constant drumbeat about Unis only considering candidates with money or Glamour not of interest here?

            On what confidence do you base your doubt?

    • AcademicLurker says:

      they have effectively outsourced their hiring decisions to NIH.

      I suppose that's the next logical step after they outsourced their tenure and promotion decisions to NIH...

    • drugmonkey says:

      In my opinion, the K99/R00 solved a problem that didn't exist

      You weren't paying attention.

      • qaz says:

        What wasn't I paying attention to? People were getting faculty jobs. They weren't getting their R01s renewed and they weren't getting tenure. But the top people each year were getting jobs.

        The goal of the K99/R00 was to decrease the time for top people to get faculty jobs. I stand by my statement that no one has ever provided any evidence that the K99/R00 changed the likelihood (or speed) of top prospects getting faculty jobs.

        And I stand by my statement that it severely hurt the not-quite-top-prospects who stumbled. (Perhaps "stumbled" is not the right word - what I mean is people who did not go to the "best graduate school", get labeled as "a top prospect" and go straight to postdoc in a "star lab". People who took some time off after graduate school, people who got bad advice and went to a poor postdoc advisor, people who switched fields, people who had trouble finding their scientific footing, etc.)

        • DrugMonkey says:

          Iirc we've established pretty convincingly in the past that you have a monolithic and distinctly old fashioned view of the proper hiring path and career arc. We have also established that I do not.

          I am more concerned about the less traditional routes.

          Not all of the K99 awardees from my Dept would have ended up in the jobs they are ink, if things were like when I was a postdoc.

          I question your assertion that the K99/R00 only serves as a marketr for the same-old elite postdocs as would be marked previously. Because I know of. Many cases that violate this assumption.

          (It is easy to say after the fact that obviously this awardee was the one if you don't consider their competition.)

  • DrugMonkey says:

    Peer review continues to be the primary input on deciding who gets funded. "the NIH" to which hiring is being outsourced is us.

  • Anonymous says:

    I, too, think that the K grant is solving the wrong problem. The real problem is why it takes someone another 4-5 yrs as a postdoc in order to be competitive for a faculty position in the biomedical sciences. This is clearly not the case in other fields -- it's even not true of bioengineering.

    This K award should recognize exceptional grad students, who, with another 2yrs of mentored training, would then be ready for a faculty position. I am in my final year of grad school, operating at the interface of statistics and biomedical science. I think the latter could benefit from having more people like me in the field. But I could be faculty somewhere in another 1-2 yrs in a statistics or engineering dept. So it's unlikely that I will continue on in biomedical science, even though that would be my preference.

    • datahound says:

      Decreasing the time from receiving a doctorate to having a faculty position and becoming an independent researcher was the initial goal of the discussions that led to the K99-R00 program. A major challenge is that there are such a large number of scientists in biomedical fields with extensive postdoctoral experience competing for faculty positions that it is difficult (but not impossible) for those with less experience to compete. The situation may be more favorable for those with your background in statistics (given the needs in modern biomedical research) so I would urge you to keep your eyes/options open and follow what most interests you.

  • […] a recent post, I examined the NIH K99-R00 (Pathway to Independence) program. Looking at two cohorts of […]

  • […] about how the program is administrated. For example, for the two cohorts of K99 awardees that I have been examining, the number of K99 awardees ranges from 1 for the National Institute (then National Center) for […]

  • datahound says:

    Drugmonkey has a post up about Sally Rockey's post in response to this discussion. http://scientopia.org/blogs/drugmonkey/2014/08/11/women-in-the-r00-phase-dont-apply-for-r01s-as-frequently-as-men/

  • […] to R00 awards and then to R01s and other awards. I have previously posted analysis including the transitions to R00 and R01 grants, gender disparity in R01 transition probabilities, differences between NIH institutes and centers, […]

  • […] analyses on the K99-R00 program, particularly the FY2007 cohort (the first year of the program). In my initial post on this program, I noted that recipients of F32 awards during the same period might make a reasonable group to […]

  • […] had previous done some analysis of the NIH K99-R00 program for the first two cohorts.  I wrote R scripts to assemble information […]

Leave a Reply