A View from the Perspective of a National Academy of Sciences Section on R01 Funding Outcomes

Nov 14 2014 Published by under Uncategorized

Throughout discussions of the NIH peer review system, it has been clear (and not at all surprising) that different individuals and groups have quite different perspectives on the NIH enterprise. This goes back to the "Enhancing Peer Review" process (and well before) and continues to the present with changes in policies and active discussions in a scientific society membership magazine (here,  here, and here) and in blogs (e.g. here),. For example, scientists who are early in their careers at present, in general, have different experiences, perspectives, and concerns than those of more established investigators. Much of the data that are presented reflect the entire NIH investigator pool although, in some cases, data for more limited subsets such as new investigators are discussed. Here, I analyze the outcomes for a different subgroup, namely the members of one section of the National Academy of Sciences.

The National Academy of Sciences is divided into sections, based on scientific field. The section I examined has 122 members at present. Let us first examine the characteristics of the group. The distribution of the members over the years in which they were elected to the National Academy is shown below:

Year elected histogram-2

The median for this distribution is 1995-1996.

The birth years for approximately 50% of these members were available online. In other cases, birth years were estimated from the year of receipt of his/her bachelor's degree or Ph.D. degree assuming an age of 22 for the receipt of a bachelor's degree and an age of 27 for a Ph.D. degree. Based on these estimated birth years, the median age at the time of election was 51 years old with a range of 38 to 71.

Of the 122 members, 108 had received substantial funding from the NIH extramural program during the period covered by NIH RePORTER (1991-present), 8 are or were members of the NIH intramural program and 6 did not receive substantial NIH funding during the period covered by NIH RePORTER.

The age distributions of research-active and research-inactive scientists is shown below:

Age distribution plot-2

Of the research-active investigators with estimated ages over 76, three are funded by the NIH extramural program, three are in the NIH intramural program, and one is active in industry.

Examination of the R01 funding records of the 108 investigators with extramural funding revealed some striking characteristics. Many of these investigators have had long-running R01 grants with the median for each investigator's longest running grant (largest suffix) of 26 years with a range of 4 to 49. 64 of 108 (59%) of the extramurally funded investigators had received one (or more) MERIT (R37) awards (extending an R01 grant for an additional cycle without a competing renewal) during the course of their career. 27 of 108 (25%) of these scientists have been HHMI investigators with all but two of these currently in such positions.

How have these investigators fared with regard to having applications funded in their initial (A0) versus amended (A1, A2) submissions? I recently posted data showing the percentage of A0, A1, and A2+ applications in the NIH-wide funded R01 grant pools from 1991 to 2014. The results for competing renewal (Type 2) grants for the 108 National Academy investigators are compared with the NIH-wide curves below:

NAS-Type2 graph

The average percentage of funded R01 grants in the A0 pool for the National Academy members averaged 94% from 1991 to 2003 compared with 60% for all NIH investigators. The A0 curve for the National Academy members roughly follows the NIH-wide curve with a dip from 2004 to 2011. Overall, these two curves show an average difference of approximately 35%.

The results for new (Type 1) grants are shown below:

NAS-Type1 graph

These data are somewhat noisy since those from the National Academy members come from only 130 new R01 grants. Again, however, the pool of R01 grants from the National Academy members include more A0 awards. From 1991 to 2003, the average percentage of A0 awards was 80% compared with 57% for the NIH-wide pool . The percentage of A0 awards for the National Academy members dipped from 2005 to 2008 and then recovered, tracking the NIH-wide curve. Overall, the average difference between the two A0 percentage curves is 30%.

The members of the National Academy of Sciences examined are a remarkable group of scientists, with important contributions throughout their careers and impressive recognition of their work including several Nobel Prizes. The purpose of this analysis was to put some quantitative context around the differences in experiences of these investigators with an average member of the overall NIH R01 grantee pool.

It is very important to note the positive feedback loops at work here. Success in getting funded and doing exciting science makes it easier to get additional funding without as much time commitment to grant writing. The time saved can then be spent doing more and better science, feeding back into the process again.


9 responses so far

  • Drugmonkey says:

    "Positive feedback loop" still implies intrinsic worthiness, Datahound. It is an equally valid hypothesis that many, many other individuals would have produced equivalently if they had a grant funding cakewalk such as you are describing. In essence, that there was nothing particularly special about them as individuals.

    (As there should not necessarily be any need for in empirical sciences.)

    • datahound says:

      DM: It does not imply intrinsic worthiness. It just means that once you get into the right set, it becomes easier to stay there (compared to others trying to move "up").

      The question of whether others would have produced equivalently given the same opportunities is a separate one. This is, of course, quite a central with regard to policy and I would very much like to have good ideas about how to approach it.

      I do not think it is fair to say these folks have done so well because they are intrinsically special (especially given the data about how the accumulated advantages relative to the general investigator population). On the other hand, I do not think it is fair to dismiss their accomplishments as being entirely due to their circumstances and not other factors.

  • drugmonkey says:

    I think we have equal evidence for each hypothesis therefore either is as fair as the other.

  • Established PI says:

    I think there is plenty of evidence against equal distribution of grant money irrespective of productivity or creativity. The widespread mediocrity in some European countries that previously lacked highly competitive grant systems is a good example. It is true that the rich get richer in any system that rewards success, and PIs who happen to get lucky with well-timed results gain momentum, attracting further funding as well as talented postdocs (with their own fellowships) that keep the cycle going. But there are also plenty of PIs who are just not successful because they don't quite have what it takes. The NIH has to strike a balance of keeping the good science flowing out of the productive labs while providing enough money to new (or relatively new) investigators so that the next cadre of creative scientists can gain a foothold.

    • datahound says:

      I do not think anyone (certainly not me) was arguing that equal distribution of grant money independent of productivity or creativity is a good plan. Competitive systems are definitely necessary to bring out the best.

      You are right about the key challenge...keeping productive labs going and comfortable enough to take risks while providing an entry ramp for a broad set of new investigators. This means giving new investigators a fair shot including sufficient funding through one mechanism or another and time to show their stuff.

  • Mikka says:

    Jeebus, who is the 98 year old in figure 2? Hats off!

    • datahound says:

      He is in the NIH intramural program. He published two papers in 2013.

    • Amboceptor says:

      It could be Hilary Koprowski (1916-2013) who was still a faculty at Thomas Jefferson University at the time of his death.

      Feel free to delete this comment if personally identifying members of the data set goes against the goals of this great blog.

Leave a Reply