Number of New and Competing Renewal Awards from 1995-2014

(by datahound) Feb 26 2015

In my recent post, I noted in passing that the number of Type 2 (Competing Renewal) awards (R01s and R37s) fell from 2653 in FY1995 to 1532 in FY2014. This led to both comments on the post and a post on this topic from Drugmonkey. Since I was also struck by this observation, I was already working on additional analysis.

Below is a plot of the number of New (Type 1) and Competing Renewal (Type 2) awards (just R01s this time for simplicity) as a function of time from FY1995 to FY2014.

Type 1-2 Award number graph

The first striking observation is the dramatic increase in New (Type 1) awards from FY1997 to FY2000 (at the beginning of the NIH "budget doubling" with no corresponding increase in Type 2 awards. This lack of increase in Type 2 awards is almost certainly due to the lack of an increase in applications although I have found no readily available data from these years. Note, further, that success rates for Type 2 applications were likely around 50% (or perhaps above) during this period (see below for data for later years).

From NIH RePORT Funding Facts, data are available for the number of applications and awards for Type 1 and Type 2 R01 grants from FY2001 to the present. Note that these data different slightly from those above and do not appear to include awards made associated with the Recovery Act. These data are plotted below.

Type 1-2 Apps Award Plot-2

This plot shows the further increase in Type 1 Applications over this period. As shown here and in the first figure, the number of Type 1 Awards has been relatively flat (after the 75% increase just prior to FY2000. The number of Type 2 applications increased gradually from FY2001 to FY2006 (by 35%), slowly fell from FY2006 to FY2010 (by 15%), and then fell somewhat more dramatically (by 25%) from FY2010 to FY2014. The number of Type 2 awards decreased by 11% from FY2001 to FY2006, by 4% from FY2006 to FY2010, and then dramatically (by 31%) from FY2010 to FY2014.

These trends are reflected in success rates for Type 1 and Type 2 R01 grants over this period shown below:

Success rate plot-2

 

The success rates fell dramatically shortly after the end of the "budget doubling" and then stabilized to some extent from FY2007 to FY2014.

Taken together, these data reveal that there has been a sharp drop in the number of Competing Renewal Awards, particularly over the past 4 years. This have been driven in large part by a drop in the number of Type 2 applications. This, in turn, may be due to the "No A2" policy or to changes in application behavior around and after the Recovery Act.

2 responses so far

Year Distribution of Competing Renewal (Type 2) Grants from 1995-2014

(by datahound) Feb 25 2015

R01 grants may be renewed, typically every 4-5 years. These are called "competing renewals" or "Type 2" grants. In the context of discussions of the "Emeritus Award" discussion, I examined the distribution of R01 grants that had been renewed over a long period of time. Here, I look at the distribution of Type 2 grants over the period from 1995 to 2014.

Data for Type 2 R01 grants (as well as the corresponding but much smaller number of R37 MERIT awards) for each year were compiled from NIH RePORTER. Note that these grants only include grants that were competitively renewed in a given year and not non-competing continuations (Type 5 awards). The distributions of the 2653 Type 2 awards from FY1995 and 1532 Type 2 awards from FY2014 over the Support Year are shown below:

1995-2014 plot

The distribution for FY1995 shows peaks at Year 4, Year 6 corresponding to renewals of initial (Type 1) grants of 3 and 5 years, respectively. There are additional peaks at Year 9 and Year 14. These correspond primarily to grants that initially made for a period of 3 years and then were renewed for 5 year periods.

The distribution for FY2014 is similar but shows some important differences. First, the overall amplitude is smaller as there were 58% as many Type 2 R01 grants awarded in FY2014 compared to FY1995. Second, the FY2014 shows peaks at Year 6 and Year 11. These correspond to grants awarded initially for a period of 5 years and then renewed for an additional 5 year period. Third, the FY2014 distribution shows a longer tail extended to Year 40 and beyond, corresponding to long-running grants.

The distributions are shown in normalized and integrated form below:

1995-2014 frac plot

These curves show more clearly the tail extending longer grant durations. For FY1995, 20% of the grants are in Years 13 or beyond whereas for FY2014, 20% of the grants are in Years 18 and beyond.

The distribution of Type 2 grants can be seen more clearly by looking are the distribution summed over all of the years from FY1995 to FY2014 as shown below.

Overall Type 2 distribution

This shows peaks from Years 5-6 and Years 10-11 and then the tail extending from Year 15 and beyond.

The structure of this tail can be seen more clearly by replotting the data on a log (base 10) scale as shown below:

Log plot-2-60

The portion of this graph corresponding to the extended tail is very nearly linear with a slight downward curvature. This indicates the tail that is approximately exponential. Fitting this curve reveals an exponent of approximately -0.064/year. This corresponds to a half-life 0f 4.7 years. In order words, the chance that a long-standing grant is renewed every 4-5 years is approximately 50%. The NIH reported success rate for all competing renewal Grants averaged 38% from FY2001 to 2014. Thus, it appears that the likelihood that a longstanding R01 is competitively renewed is slightly, but not dramatically, higher than that for R01 grants overall. The slight downward curvature of the log plot likely reflects that fact that there were a smaller number of longstanding grants in the earlier years in this analysis.

An alternative approach to examining trends the duration of these grants involves looking at the fraction of A0 applications among the funded Type 2 grants. I have previously examined this parameter in other contexts. A plot of the fraction of A0 applications among funded Type 2 grants as a function of the Year of the grant is shown below:

Fraction A0 plot

 

This fraction dips slightly for grant years from 4-7 (corresponding to the first renewal) and then reaches a relatively stable level extending out to 40 years. This suggests there is not a major increase in the likelihood of application success as the year of the grant increases.

Overall, these observations are consistent with the notion that R01 grants reach longstanding status through the perseverance of the principal investigators. Over time, these investigators continue to execute research programs that are sufficiently productive that they compete for renewal with a success rate close to 50% (at least historically). There does not appear to be a substantial advantage for longstanding grant applications above the general advantage for Type 2 versus Type 1 applications based on these publicly available data.

15 responses so far

"Age and the Trying Out of New Ideas"-Initial Impressions

(by datahound) Feb 18 2015

Alerted by a post on Nature News and Comment, I read with interest a newly posted paper from Mikko Packalen and Jay Bhattacharya from the National Bureau of Economic Analysis entitled "Age and the Trying Out of New Ideas."

The abstract of this working paper states:

Older scientists are often seen as less open to new ideas than younger scientists. We put this assertion to an empirical test. Using a measure of new ideas derived from the text of nearly all biomedical scientific articles published since 1946, we compare the tendency of younger and older researchers to try out new ideas in their work. We find that papers published in biomedicine by younger researchers are more likely to build on new ideas. Collaboration with a more experienced researcher matters as well. Papers with a young first author and a more experienced last author are more likely to try out newer ideas than papers published by other team configurations. Given the crucial role that the trying out of new ideas plays in the advancement of science, our results buttress the importance of funding scientific work by young researchers. (Emphasis added)

Needless to say, I was intrigued. After a quick read, I looked deeper into the methodology, particularly with regard to the highlighted terms above.

The study is based on the use of MEDLINE (accessed through PubMed). More precisely, they used “Author-ity” MEDLINE, a previously constructed version of MEDLINE with the names of authors disambiguated as much as possible. This database was used for two purposes. First, new ideas were identified by searching titles and abstracts for two- or three-word strings and associating these with the year when they first appears. Strings that subsequently occurred with high frequency were deemed to be important new ideas. The Nature commentary includes a list of the ten most frequent concepts for each decade and inspection reveals these to be sensible. Second, the "age" of each investigator was estimated by determining the year in which the first publication by this investigator appeared. Thus, this is "career age" rather than chronological age. This is a sensible approach which has both advantages and disadvantages. Most importantly, it is workable from the available data. I know from some of my recent analyses, estimating chronological ages of investigators can be quite difficult. In addition, this automatically at least partially corrects for increasing training periods over time. A disadvantage is that an early publication can "age" an investigator compared to peers.

With these two parameters, the authors were set to do some analysis. Figure 1 in the paper is shown below:

Screen Shot 2015-02-18 at 8.28.34 AM

Panel A shows the fraction of publications trying out new ideas versus the career age of the first author. Clearly, there is a broad peak in which the first authors are within 3-12 years of his/her first publication. This, of course, primarily reflects the accomplishments of graduate students and postdocs!

Panel B shows the comparison for All Authors. This shows a more featureless downward trend. This probably reflects the contributions of graduate students and postdocs but with more senior members of the research teams in the mix.

Panel C shows the distribution with Key (both first and last) Authors. This shows a peak in the career age range of 10-15 years. Given the first author distribution from Panel A, this suggests that the last author distribution has a peak around 20 years. This is confirmed by the data presented in Figure 3 which shows the data in two-dimensional format with Career Age of Last Author versus Career Age of First Author.Screen Shot 2015-02-18 at 8.37.45 AM

 

The maximum for this post occurs with first authors with career ages between 1 and 10 years (i.e. graduate students and postdocs) and last authors with career ages between 8 and 25 years (early to mid-career faculty).

Does this mean that early to mid-career investigators are the most productive in trying out new ideas? Yes and no. As a population, they certain do appear to be. However, these data have not been normalized (as far as I can tell) to the distribution of the ages of investigators. This distribution (which has been changing over time as we were recently reminder by Drugmonkey) shows peaks (both for R01 grantees and medical school faculty) in the range of 45-55 years old over the period covered by this analysis.  If one assumes an age of the beginning on independent careers of 36 (over this period not just at present), the data are consistent with the number of faculty at each career stage being an important factor.

Overall, the paper clearly supports the roles of graduate students and postdocs in being first authors of many (most) papers that appear to break new ground. This is, as George Carlin would say, "near-fetched." The results regarding last authors will require a more careful reading of the paper and, perhaps, more analysis by the authors. However, this is clearly an important data set and approach to provide empirical evidence that bears on these important issues.

6 responses so far

Longevity and Transitions in in R01s in Years 40+...Part 2

(by datahound) Feb 10 2015

I realized that my previous analysis was missing a key bit of information, namely how many long-standing R01s from previous years failed to make it to the present. I examined R01s in years 40+ from FY2010. There were 47 grants awarded to 47 distinct PIs. Of these grants, 24 do not appear as active, funded awards at present. Thus, approximately 50% of the R01s in years 40+ in FY2010 are still funded at present and 50% are not.

Of the PIs corresponding to the 24 year 40+ R01 grants that are no longer funded, 9 still have other NIH funding at present. In most cases, these are other long-standing (but less than year 40) grants while in a few cases they appear to be new projects.

One response so far

Should grants be limited to a single renewal?

(by datahound) Feb 10 2015

In the context of the discussion of the "Emeritus Award" from NIH, Neuro-conservative commented:

I am curious what you (and others here) would think about limiting grants to a single renewal, or any other limitations on duration? I previously thought it reasonable that renewal of ongoing solid work should be slightly favored within the system. But I think that Prof. Rosenbaum has inadvertently persuaded me otherwise.

Thoughts?

13 responses so far

Longevity and Transitions for R01s in Years 40+

(by datahound) Feb 07 2015

In the context of the potential "Emeritus Award" discussion, two of the points on interest were (1) an understanding of the situations of the senior investigators to whom such an award mechanism would be presumably targeted and (2) the fact that mechanisms already exist for transitioning labs to more junior faculty if that is desired. To get a look at one aspect of this, I examined active R01 grants in years 40 or larger. Of course, this is an atypical slice of this pie as many investigators, even if they have been continuously funded for decades, have not done so on individual grants that have been renewed.

I identified 62 active R01 grants in years 40-58. These were held by 59 investigators (three investigators each had two R01s on the list). Seven of the grants included co-PIs. The ages or year of degree could be identified for most investigators through internet searches. For 13 of the grants, it appeared that the grants had been transferred from another PI at some stage of its existence. In two cases, this appeared to be due to the death of the original PI. In seven cases, the point of transition could be identified and the original PI could be identified and all appear to be still alive. In these cases, the ages of the original PIs at the time of transition were estimated to range from 56 to 86 with a median of 74 while the ages of the PIs to which the grant was transferred were estimated to range from 42 to 65 with a median of 50. In the remaining four cases, the point of transition could not be identified, but the current PIs did not appear to be old enough to be the original PIs.

Overall, the ages of the current PIs for these grants are estimated to range from 49 to 93 with a median of 74. The ages at which the original PIs were awarded these grants were estimated to range from 24 to 40 with a median of 32.

21 responses so far

Request for Information: Potential Emeritus Award for Senior Researchers

(by datahound) Feb 04 2015

Even though I have heard discussion of the concept over the years, I must admit I was a bit stunned to see the Request for Information (RFI) from NIH regarding a "Potential Emeritus Award for Senior Researchers". The introduction for this RFI reads (in part):

An important issue for NIH is the long term succession planning for the research we support.  Over the years, NIH has been persistent and creative in efforts to support early career investigators through policy changes and new programs.  But we must also consider the needs of our senior investigators and how NIH can assist with the continuation of their well-established research programs, should they wish to transition to new positions.  While many senior investigators may be happy pursuing their research questions in the laboratory, others may be looking to move into other roles, such as full time teaching and mentoring.  Our senior investigators have invested their careers to establish the intellectual and technical infrastructure needed to pursue their research questions, and even if they wish to pursue new roles, they may not wish to dismantle their long-standing programs.

I find many aspects of this request surprising. These include:

(1) This problem already has a solution. An investigator can (with approval from the relevant IC) name a new Principal Investigator for a grant. Assuming the PI is qualified and NIH approves, this is an effective transition strategy that has been used many times.

(2) For most research programs, is "succession planning" something that NIH staff are worried about? Given that many investigators train numerous younger scientists over the course of their careers and that the system is currently flooded with accomplished younger scientists, the solution to this problem without any mechanism seems to be at hand.

(3) Even proposing such a mechanism seems quite inappropriate and tone deaf at this juncture when so many younger scientists are struggling to establish and maintain their careers.

Let me add two more personal observations. First, as someone who has changed roles several times over the course of my career, I know it can be done without any formal mechanism. Changing career directions can be a bit scary but I have been blessed with some tremendous opportunities and am glad that I have followed combinations of my heart, my brain, and my family to pursue them. I have developed many new skills and have had the privilege of going through the tenure process four times. In my experience, you just have to try to do the right thing, for yourself, your family, and your communities.

Second, when I was at NIH, I discovered that a senior and very accomplished faculty member had not tried to renew his R01. I emailed him and asked what was up. He said 'I have sources for some other funding and it is time to give someone else a turn'. I have considerable admiration for many senior scientists who have accomplished much over the course of their careers, but there does come a time when it is time to give someone else a turn.

NIH's requests information about:

  • Community interest in an emeritus award that allows a senior investigator to transition out of a role or position that relies on funding from NIH research grants
  • Ideas for how one would utilize an emeritus award (e.g., to facilitate laboratory closure; to promote partnership between a senior and junior investigator; to provide opportunities for acquiring skills needed for transitioning to a new role)
  • Suggestions for the specific characteristics for an emeritus award (e.g., number of years of support; definition of a junior faculty partner)
  • Ways in which NIH could incentivize the use of an emeritus award, from the perspectives of both senior investigators and institutions
  • Impediments to the participation in such an award program, from the perspectives of both senior investigators and institutions
  • Any additional comments you would like to offer to NIH on this topic

I hope you all will take advantage of the opportunity and share your thoughts. I certainly plan to.

42 responses so far

The Tension Between Average Grant Size and Success Rate

(by datahound) Feb 01 2015

Once of the most challenging and contentious issues that is faced by the leadership of NIH institutes and centers involves policies about cutting requested budgets for R01 grants. Once the amount of funds available for new and competing grants is known, the first question is how to divide the pie.

One side of the argument is that making as many awards as possible is crucial since having a funded grant can be the difference between facilitating the launch of a promising career for a new faculty member or shutting down a lab even if it is productive.

The other side of the argument is that requested budget reflect the real costs of the proposed research and that cutting requested budgets substantially can result in systematically underfunded grants for which the budget cannot support the necessary research efforts. This situation may be exacerbated by the modular cap which has remained unchanged since it was created and evidence indicates that PIs may be requesting $250K at the cap rather than requesting a larger budget and running the risk of suggested budget cuts from the study section.

Some policies come into play. First, NIH policy requires that a PI be allowed to modify his/her specific aims if the budget is cut by more than 25%. This plays a role in limiting the size of cuts and also has led to a large number of R01s awarded at the level of $188K (since $250K (modular cap) X .75 =187.5K. Second, Congress often includes language associated with appropriations bills that indicates that, for example, average grant sizes must be the same as in the previous year. This can limit flexibility within NIH institutes and centers.

As the "NIH Doubling" ended, it was easier to make the case that cutting grant sizes a bit while keeping more labs funded made sense since hope sprang eternal that budget increases larger than BRDPI were coming soon. However, as the decade of flat budgets continued, followed by the sequester, this became more problematic. The net results was that the buying power (size increase corrected for BRDPI) of the median R01 grant fell by 18% from 2003 to 2013.

What steps should be taken now? Should the modular cap be increases (or dropped altogether)? Should cuts to grants when they are awarded be minimized? Should cuts to non-competing grants be discontinued? Other suggestions? Bear in mind that the laws of arithmetic still apply and any increase in average grant size will result in a decrease in the number of grants that can be funded.

31 responses so far

Estimated Publication Outputs as a Function of Number of R01 Grants per PI

(by datahound) Jan 30 2015

In a recent post, I presented the distribution of the number of investigators who were PIs of from 1 to 6 R01 grants in fiscal year 2014.

The significance of this distribution is hard to access without some knowledge of the outputs from these grants. This is a challenging problem. Here, I attempt to estimate some of these outputs. The approach I used in very much an approximation, but some important trends can be discerned.

To estimate outputs, I collected the results in PubMed for publications from 2012 to the present for the 7 PIs with 6 R01s, the 31 PIs with 5 R01s, and randomly selected samples of 30 PIs with 1-4 R01s (screened only for name ambiguity issues or the present of large amounts of non-R01 funding).

The distributions of the number of publications over this period as a function of the number of R01s is shown below:

Pub Number Plot-250

The median number of publications for the sample of PIs with 1 R01 is 11.5. This doubles to 23.5 for the sample of PIs with 2 R01s. Essentially the same median was found for the sample of PIs with 3 R01s. The median number then increases monotonically to 44 for the PIs with 6 R01s.

In addition to the trends, it is important to note the substantial variations that occur at each level of funding. This reflects difference between publication patterns in different fields as well as the performance of individual PIs.

These data are re-plotted after normalizing the number of publications by the number of R01 awards below:

Pub Ct Normalized plot

This graph shows that the median number of publications per R01 grant is essentially constant at 11.5 for PIs with one or two grants and then drops to approximately 7.5 for PIs with 3-6 R01s.

A criticism of my earlier analyses of this type has been the lack of separation of publications in high impact journals (for this purpose Science, Nature, and Cell) because of the presumed large cost of generating such publications. The numbers of such publications as a function of PI from the assembled publication database were determined and added to the plot below.

Pub Count with CNS

PIs have between 0 and 16 Science, Nature, and Cell publications over this period. Overall, 40% of the PIs have at least 1 such paper including 17% of the PIs with a single R01 and between 37-57% of the other groups. The number of Science, Nature, and Cell publication is modestly correlated with the total number of publications with a correlation coefficient of 0.26.

The average number of Science, Nature, and Cell papers versus average total number of publications for the six groups of PIs is plotted below:

Fraction of CNS paper plot

A nearly linear relationship is observed with a slope of approximately 3.5 Science, Nature, and Cell papers per 100 total publications. From these samples, this proportion does not vary dramatically as a function of the number of R01 grants.

Again, this is a rough estimation to much many caveats apply. First, of course, publication numbers without further analysis, is a limited measure of true productivity. Second, the publication records are assembled over a period of time over which funding likely varies to some (and differing) extents for each PI. Third, relatively small samples were used for the groups with 1-4 R01s. Nonetheless, I hope these data will provide a framework for discussions about the implications of the current distribution of R01 resources across PIs and those of potential alternatives.

UPDATE:

In response to the comment from Drugmonkey, I have plotted the number of publications normalized to the number of R01 grants with each Science, Nature, and Cell paper counted as 5 publications below:

Pub Count-CNS-5 graph

 

This graph looks very similar to the previous one while such papers were counted as single publications. This similarity is to be expected given that the fraction of Science, Nature, and Cell papers out of total publications does not depend substantially on the number of R01 grants as shown by the approximately linear relationship above.  Indeed, this graph looks similar even when Science, Nature, and Cell papers are weighted as 10 publications.

4 responses so far

R01 Size Distribution for PIs with a Single R01/R37 Grant in FY2014

(by datahound) Jan 28 2015

A comment to my previous post asked about the size distribution for R01s for investigators with a single R01 grant. This distribution is shown below:

Single R01 histogram figure

The median is $371K (total costs) with an interquartile range of ($464K - $318K) = $146K. This distribution is very similar to that for all R01s for FY2013 presented in an earlier post.  Approximately 1/3 of the grants fall outside of a Gaussian distribution centered around the mode of the distribution. These represent grants that are outside of the range determined by the modular cap.

UPDATE:

Here is a plot of the cumulative fraction of total R01 funding going to PIs with single R01 grants as a function of R01 size from smallest to largest:

Single R01 Cum graph

 

Grants up to the modular cap limit of approximately $375K total costs account for approximately 55% of the total R01 expenditure for PIs with single R01 grants.

2 responses so far

Older posts »