Few science policy topics have led to as much discussion as the NIH policy with regard to the number of amendments allowable for grant applications. This policy originated from the NIH Enhancing Peer Review draft report. Figure 8 in this report shows that the fraction of R01s that were funded in their original submission (A0) had dropped from more than 60% during the doubling years to less than 30% in 2007 with a concomitant increase in the fraction of A1 and, particularly, A2 applications that were funded. These data, some other analyses, and anecdotes from study sections, suggested that study sections were "queuing" applications, giving better scores to A2 applications since these was these application's "last chances."
The Enhancing Peer Review report proposed "consider(ing) all applications as new" so that there was no reason for study sections to treat applications differently based on their amendment status. Organizations including FASEB did not support this proposal. In order to help decrease the time between the initial submission and the time of funding for applications that would eventually be funded, the NIH countered by eliminating the option of A2 submissions.
From my perspective, this proposal would have been sensible in times of reasonable paylines. However, since this proposal was adopted, paylines have continued to fall (except during the ARRA years) to the point that the variability in scores means that two chances may not be adequate, even for outstanding proposals. I wrote a column about this for ASBMB today entitled "On deck chairs and lifeboats".
After resisting pressures from many in the scientific community for years, the NIH recently reversed course and now will allow an application that did not get funded in A1 form to be resubmitted as a new A0 application without restriction. This policy is a hybrid of the old policy and the initial proposal from the Enhancing Peer Review report. To my knowledge, NIH has not provided much guidance into the analysis that led to this reversal other than statements of concern about the impact of the "No A2" policy on early stage investigators.
As I thought about this policy, I realized that I had never seen data about the differences between new (Type 1) and competing renewal (Type 2) applications. Type 2 applications, in general, score better that Type 1 applications due, in part, to the fact that only applicants that have been relatively successful submit renewal applications. The differences can been seen in an NIGMS Feedback Loop post that revealed that (for the January 2010 Council round in FY2010) 50% of the Type 2 applications scored better than the 20th percentile compared with less than 30% of the Type 1 applications, even those from established investigators.
Below are plots of the number of NIH-wide R01 awards for Type 1 and Type 2 awards as a function of amendment status for FY2003-FY2013:
The curves showing the mix of A0, A1, and >A1 (almost all A2) applications for Type 1 and Type 2 awards are remarkably similar, with comparable rises in the fraction of A2 awards from FY2003 to FY2009, followed by a drop to essentially zero due to the "no A2" policy. The results for Type 1 and Type 2 awards are compared directly in the plot below which shows the fraction of A0 awards out of total awards for each type.
In FY2003, the fraction of A0 awards among Type 1 grants was slightly larger than 0.5 whereas that for Type 2 was higher at approximately 0.6. As these fractions dropped, rose, and then dropped again, these two fractions are more equal near 0.45 in FY2013.
Closer examination of the first graph reveals another important point. In FY2003, there were 4564 Type 1 awards and 2618 Type 2 awards. By FY2013, these had dropped to 3403 Type 1 awards and 1390 Type 2 awards. Note that these are new and competing awards, not all R01 awards (which would include non-competing (Type 5) awards). The drop in the number of Type 1 awards is 25% whereas the drop in the Type 2 awards is nearly twice as large (47%). One can speculate that this drop in the number of competing renewal awards since the ARRA bump may be a driving force in concerns expressed in recent years by established investigators about the direness of the funding situation.
What accounts for these decreases in the number of awards? Of course, the NIH appropriation has not grown substantially since FY2003 and has dropped substantially when corrected for inflation. However, the average R01 sizes (total costs in nominal dollars) increased by 24% for Type 1 awards and only 16% for Type 2 awards (although the distributions should be examined to understand these increases more fully, as discussed in an earlier post). The drop in award numbers reflects that fact that the total expenditures on R01s was $9.76 B in FY2003 and was $9.80 B in FY2013 whereas the overall NIH appropriation was $26.74 B in FY2003 and $29.13 B in FY2013, that is, the fraction of the NIH budget going to R01s has decreased over this period.