The Recovery from the Recovery Act-Part Deux

May 05 2014 Published by under Uncategorized

In a previous post, I examined the awards made as part of the American Recovery and Reinvestment Act (ARRA) and sorted them according whether these awards were made to investigators who had other funding in FY2008 ("Existing") or to investigators who did not have funding in FY2008 ("New"). I have now repeated the analysis in terms of investigators rather than awards, have examined the funding status of these investigators prior to receiving ARRA funding in a more refined way, and have investigated the funding status of the investigators subsequent to FY2010 when ARRA funding terminated.

The key results from this analysis are as follows:

5182 investigators received 1 or more non-supplemental ARRA award.

Of these:

2526 were funded with some NIH in FY2008

449 were funded in FY2007 but not FY2008

273 were funded in FY2006 but not 2007 or 2008

1934 were not funded in FY2006, FY2007, or FY2008. Some of these appear to be "New Investigators" by the NIH definition while others either have had R-mechanism funding in the past or have been funded by other NIH mechanisms.

Further examination of these groups revealed a substantial difference in the mechanisms by which these investigators were funded with ARRA funds.

ARRA Mechanism plot

More than 40% of the investigators who had been funded in FY2006, FY2007, or FY2008 received an R01 award through ARRA. The majority of these are likely 2-year R01s although the distribution of these awards between 2-year R01s and longer R01s with additional years funded through non-ARRA funds was not investigated further. In contrast, less than 20% of the awards to investigators who had no R-funding from FY2006-FY2008 were R01s. Instead, this group of investigators were funded through a mixture of mechanisms with smaller budgets including R21s, R03s, and R15s. The fraction of ARRA-specific RC1 ("Challenge Grants") and RC2 ("Grand Opportunity Grants") was approximately the same across all four groups.

With these lists of investigators available, it was possible to examine funding subsequent to ARRA. Overall, 2418 of the 5182 investigators were funded in FY2011 (47%). This breaks down as follows:

1760 of 2526 investigators who were funded in FY2008 (70%)

174 of 449 investigators who were funded in FY2007 (but not FY2008) (39%)

88 of 273 investigators who were funded in FY2006 (but not FY2008 or FY2007) (32%)

396 of 1934 investigators who were not funded in FY2008, FY2007, or FY2006 (20%)

In FY2012, 2532 ARRA funded investigators were now funded with 2109 of those also funded in FY2011, 423 newly funded in FY2012, and 309 funded in FY2011 but not FY2012.

This analysis reveals approximately 60% of the non-supplemental ARRA awards went to investigators who were already funded or had been recently funding prior to ARRA while approximately 40% of these awards went to investigators with no recent history of R-mechanism funding. Of the later group, about 20% were refunded in FY2011. About 9% of this group were refunded in FY2012.

 

23 responses so far

  • So the vast majority of ARRA R funds that went to previously poorly NIH-funded PIs didn't help them become stably NIH-funded PIs. (Not saying that this was a goal of ICs for spending ARRA funds, but it could have been.) Another interesting analysis would be to compare the productivity of ARRA funds that went to the previously poorly NIH-funded PIs versus the previously stably NIH-funded PIs. I suspect that there will be fewer published papers per ARRA dollar spent for the former than the latter.

    • datahound says:

      I think your conclusion is both correct and not surprising. It is hard to imagine many two-year awards converting modestly competitive investigators into a stably funded investigators, particularly in this climate. The productivity study would be interesting (and I am inclined to agree with your hypothesis) but would be very hard to do, both in terms of data availability and robustness and assignment of productivity to ARRA funds as opposed to other funds available to stably funded investigators.

    • AcademicLurker says:

      nother interesting analysis would be to compare the productivity of ARRA funds that went to the previously poorly NIH-funded PIs versus the previously stably NIH-funded PIs. I suspect that there will be fewer published papers per ARRA dollar spent for the former than the latter.

      Almost certainly. Already stably funded PIs will have pre-existing momentum that the poorly funded lack.

  • drugmonkey says:

    This sort of answers the question left by your longitudinal analysis post. The In/Out churn numbers surrounding ARRA are unusual by something on the order of 2,000 PIs. Remarkably consistent with your identification of about 1,900 without recent funding.

    I'll look on PhysioProf's point as more of a glass-half-full though. The extra money allowed some newbies a *chance*. And some 20% of them leveraged this chance into a (hopefully) sustained interval of funding. Great!

    We can take this a bit farther and apply it to any initiative that you can think of that is designed to extend the chance to any category of PI . ESI, women, underrepresented groups, target scientific domains, geographical regions, University types, etc, etc. Plop down some grant funding, presumably get *some* science accomplished in the short term and some subset of those PIs will make the transition to sustained funding. Some won't. This isn't a permanent entry card. Obviously, from your data.

    We can argue about the quality of the "chance" that is extended by ARRA, ESI help, the old R29, R56s etc, but the point here is that any of the above extends an opportunity to PIs who otherwise might have had an even harder challenge. Some of them will be rightfully moved into the column of the "continuously funded".

    Another point is that ARRA may not have been as counterproductive as people mutter about around the water cooler. New PIs entered but as many or more left the building after the two year awards expired. ARRA clearly had more of an effect propping up the usual suspects than it did bringing new mouths to the trough.

    • datahound says:

      You raise a good point about the fact that a fraction of those brought in through ARRA funding may have found a relatively stable niche, at least for some time. My major concern, in this regard, relates to the use short-term funding. If the goal is to bring more new PIs into the mix, then I would hypothesize that funding a collection of 2-year grants would produce a lower yield than funding half as many 4-year grants (but this is just my intuition; I cannot point to data to support this although I will think about how one might address this). Of course, 2-year grants were mandated by the ARRA rules.

      • drugmonkey says:

        The percentage of the temporary awardees who make it to sustained funding may not be a concern. One might desire a bigger pool with more diversity of projects being extended the chance to seriously compete. For example.

        When it came to ARRA, of course, broad geographical distribution had to be even more of a plus than usual?

  • eeke says:

    Without continued funding at ARRA levels, it baffles me as to why the NIH staff would think that funding an extra number of investigators would be sustainable. As for CPP's comment, of course previously funded investigators would show greater productivity. It is assumed they already had stuff in the pipeline, whereas poorly funded investigators would not. The ARRA may give them a "bump" in productivity which is likely to only show up about now (in terms of publishing results). I would measure productivity as a result of ARRA support as a function of change from prior productivity.

    • datahound says:

      Two points: First, overall NIH staff were quite circumspect about funding too many extra investigators with ARRA funds. More than half the ARRA awards were supplements to ongoing grants and approximately 60% of the remaining awards went to funded or recently funded investigators. Second, a major challenge with policy development under these circumstances is not knowing future NIH appropriations. Hope sprang eternal that the ARRA bump would be extended, at least in part, in terms of an increased appropriation level. Furthermore, having recently funded investigators who would bring new science into the NIH portfolio would be part of the argument for such increases. Of course, this did not happen. Policy development is easier in retrospect.

  • lurker says:

    Just how helpful is the "ESI help"? OR the "XX help"? If ARRA influx was followed by a rapid efflux mainly because the poorer/younger labs didn't have already previous funding to generate data that then made them look "productive" in using the ARRA funds, this points to the real problem with the study section/peer review process being unrealistic and retrospective in their selection process for winners.

  • drugmonkey says:

    Why are you speculating on what NIH staff thought? Why are you not asking what *Congress* thought was supposed to happen? Hint: a movement on the unemployment needle. Hint 2: buying stuff, preferably from US companies.

    • eeke says:

      You're right, this falls on congress and policy and not the NIH. A pile of money being thrown at them is a good thing. It's too bad it all had to be spent in a short period of time instead of trying to draw it out (in anticipation of tight budgets, sequesters and all that). I understand it's impossible to predict future appropriations, though.

  • drugmonkey says:

    174 of 449 investigators who were funded in FY2007 (but not FY2008) (39%)

    88 of 273 investigators who were funded in FY2006 (but not FY2008 or FY2007) (32%)

    I wonder to what extent this models the usual behavior of POs in picking up awards out of order to save their long term buddies? Over a third getting back into the game after an unfortunate dry spell looks okay-ish from the PI perspective. A gap isn't a lab death sentence. OTOH, what if two-thirds of the time POs let sympathy rule their pickups, the money is wasted in terms of getting the lab back on competitive footing?

    • datahound says:

      DM: I have looked at the distribution of investigators for the cohort funded in FY2006 but not FY2007 or FY2008 in terms of the year of their first funding in RePORTER. The distribution is quite broad going back to 1986 with 10 of the 88 investigators having R29s (FIRST Awards) as their first funding. The median is 1998. About 1/4 of these investigators started major funding after 2003. Thus, this looks like a broad mix of relatively senior investigators who had a gap in R-funding (although some had substantial funding from other mechanisms) and investigators who were at a career stage that they were competing for first renewal funding.

      In contrast, the cohort with no funding from FY2006-FY2007 was more than 60% investigators who do not appear to have received any substantial NIH funding prior to ARRA (that is, NIH-defined New Investigators). The remainder of this distribution looks approximately like that described above for those with funding in FY2006.

      Some of the New Investigators presumably match the experience of Recipient in terms of the impact of ARRA funding on their short-term career prospects.

  • Recipient says:

    Interesting analysis. As it happens, I'm one of that 20% group. I started my first faculty appointment in 2005 with a K22 in hand. In 2006, I received a good (not great) sized foundation grant. These, together with my start up funds, made it so that I was quite comfortable.

    By the spring of 2008 (3 years in), I had still not yet secured R-level funding and my institution threaten to fire me. They gave me 9 months to get R-level funding, otherwise, I was done (despite having plenty of money to run my lab for at least another 2 years). Then, ARRA hit, and one of my several R01 applications which had failed (percentile in the 20s) was funded for 2 years. Shortly thereafter (weeks), I also received a DP2 and an R21. I went from almost fired to celebrated brilliant young faculty faster than you can say press release. My two year ARRA R01 was ultimately renewed in January of 2012 (through the end of 2016).

    So, yes, the ARRA R01 was a key to my successful move from assistant professor to tenured associate professor in 2012 (since the other grants were not renewable). I don't know if 20% is a good number in terms of expecting people to remain funded in this climate, but I'm very thankful for the boost the ARRA funding provided my lab and my career. Of course, it's hard to fully separate the ARRA impact given that I got other substantial money almost at the same time, but because it was the only renewable grant, it was definitely important.

    I thought you'd all like to at least hear an n=1 success story stemming from the ARRA program. It was important for me, and I'm sure the other 20% would say the same thing.

  • PI in RI says:

    Recipient, very glad to hear about your success story seeded by ARRA funds!

    I was actually about to pose a question about what the probability is of a PI making a new hire due to funding of a 2 yr grant (eg. R21) vs. a 4-5 yr grant (eg. R01)? Makes sense that longer funding period = stability = confidence to make hire. But based on Recipient's case, renewability may weigh in significantly. Of course, the renewable 2-yr grant seems to be a rare animal that had a spike in appearance during ARRA.

    Datahound, is there a way to determine what % of those shorter ARRA new R01s won competitive renewal? This appears to be a slightly different metric than the investigator level analysis of whether they had R funding or not following ARRA.

    • datahound says:

      It may be possible to sort this out without too much work, but I need to think about the best strategy. Some of these were awarded for two years and some were awarded to two years with ARRA funds and extended.

      • datahound says:

        PI in RI: It appears that there were 1144 new (Type 1) R01s funded by ARRA and an additional 672 competing renewal (Type 2) R01s funded by ARRA. Of these, 255 were extended (Type 4 Award) beyond the two years that were supported by ARRA. Of these that were extended, 223 were new (Type 1) grants going into ARRA and 22 were competing renewal (Type 2) grants going into ARRA.

        Of the total of 1144+672 = 1816 R01s, 132 have been renewed to date. Of the grants that were renewed, 40 were new (Type 1) grants going into ARRA and 92 were competing renewals (Type 2) grants going into ARRA. Thus, the overall renewal rate to date is 132/1816 = 7.3%.

        Of course, additional grants may be competing for renewal at present.

        • Recipient says:

          Only 7.3%?? That seems like a depressingly low number to me.

          • datahound says:

            I agree. This is a low number although I am not surprised that it is tough to get an R01 competitively renewed after only 1 to 2 years. It is possible that some of these investigators have been successful with other R01s subsequently, either after not being successful with renewals of the ARRA awards or after just deciding to start with a new proposal. I can try to analyze this further.

  • Recipient says:

    Thanks. And as a former member of my departmental CAPT and now a member of my institution's school-level (med school) CAPT, I can say that renewability is indeed a factor.

  • "to save their long term buddies" == "to preserve a prior investment in know-how, equipment, and idea development"

  • drugmonkey says:

    That too PP, that too.

    Sometimes.

    But either way you describe it, two thirds of the time it doesn't work. Isn't that of interest?

Leave a Reply