How long should a postdoctoral experience be from a training perspective?

(by datahound) Dec 11 2014

In light of the recent release of the National Recent Council report The Postdoctoral Experience Revisited as well as my recent analysis of the age distribution of New and Early Stage Investigators, I have been thinking again about optimal time for a postdoctoral experience. From an analysis that we did when I was at NIGMS and subsequently shared through Drugmonkey, the median postdoctoral period among NIGMS grantees who received R01s in FY2004-2006 was 5.0 years. It seems likely that this period has increased subsequently.

Many factors contribute to the length of a postdoctoral experience including training needs, career options, other life factors, and so on. Here, I would like to get thoughts on how long a postdoctoral experience purely FROM A TRAINING PERSPECTIVE. In other words, how much time in a postdoctoral experience does a young scientist need to learn new concepts, techniques, and so on without regard to the need to accumulate publications, etc. in order to be competitive for their desired career path?

33 responses so far

Estimated New Investigator Distribution for FY2014 by IRG

(by datahound) Dec 09 2014

In a previous post, I examined the distribution of new investigators across NIH Institutes and Centers. I have now performed a preliminary analysis of these data as a function of NIH Integrated Review Groups (IRGs).

The percentages of estimated new investigators as a function of the total number of awards for each IRG is shown below along with the percentage of A0 awards among the funded grants and the percentage of Type 1 (new) awards are shown below:

IRG IRG % A0 % Type 1 % New Investigator
AIDS and Related Research AARR 38 79 26
Biobehavioral and Behavioral Processes BBBP 33 73 36
Biological Chemistry and Macromolecular Biophysics BCMB 42 61 24
Brain Disorders and Clinical Neuroscience BDCN 31 71 24
Bioengineering Sciences and Technologies BST 50 73 27
Cell Biology CB 47 54 18
Cardiovascular and Respiratory Sciences CVRS 38 70 24
Digestive, Kidney and Urological Systems DKUS 38 57 24
Endocrinology, Metabolism, Nutrition, and Reproductive Systems EMNR 39 67 26
Emerging Technologies and Training Neurosciences ETTN 41 71 12
Genes, Genomes, and Genetics GGG 54 55 22
Healthcare Delivery and Methodologies HDM 43 86 42
Infectious Diseases and Microbiology IDM 41 64 29
Integrative, Functional and Cognitive Neuroscience IFCN 36 64 27
Immunology IMM 26 74 24
Interdisciplinary Molecular Sciences and Training IMST 57 71 37
Molecular, Cellular and Development Neuroscience MDCN 45 65 27
Musculoskeletal, Oral and Skin Sciences MOSS 41 70 27
Oncology 1-Basic Translational OBT 35 73 27
Oncology 2-Translational Clinical OTC 38 81 31
Population Sciences and Epidemiology PSE 39 84 30
Risk, Prevention and Health Behavior RPHB 19 94 26
Surgical Sciences, Biomedical Imaging and Bioengineering SBIB 38 68 28
Vascular and Hematology VH 36 73 22
Special Emphasis Panels (All IRGs) SEP 58 76 23
MEAN 40 71 27
MEDIAN 39 71 26

 

The percentage of new investigators ranges from 12% to 42%. However, the 12% figure should be viewed with caution since relatively few R01 grants were funded in the ETTN IRG. Also, it is important to recall that these are "new investigators" and not necessarily "early stage investigators". Thus, the relatively high percentage of new investigators in the HDM IRG may represent investigators new to NIH rather than early stage investigators.

The three percentages (%A0, % Type 1, and % new investigators) might be expected to related to one another. The %A0 and % New Investigators are essentially uncorrelated. However, the %Type 1 and % new investigators are correlated (correlation coefficient 0.43) indicating that those IRGs that fund a higher percentage of Type 1 awards tend to fund a higher percentage of new investigators. This almost has to be true since an IRG that funds a higher percentage of Type 1 awards, funded a lower percentage of Type 2 (competing renewal) awards and Type 2 awards, by definition, do not go to new investigators.

No responses yet

Twelve Months of Datahound (Eight Month Edition)-2014

(by datahound) Dec 04 2014

Following Drugmonkey...It's been a good start.

April     Datahound's first post...

May     Indirect cost rate survey:  Motivated by the new post, I finished the list for these 49 institutions...

June     Few science policy topics have led to as much discussion as the NIH policy with regard to the number of amendments allowable for grant applications.

July     Gender disparity in K99/Roo awards...Of the 201 men with R00 awards, 114 (57%) have gone on to receive at least 1 R01 award to date. In contrast, of the 127 women with R00 awards, only 53 (42%) have received an R01 award. This difference is jarring and is statistically significant (P value=0.009).

August      These data reveal that the number of applications increased more than 2.6-fold from 5283 in FY2003 to almost 14000 in FY2012. NIH responded to this "proposal pressure" by increasing the number of awards from 1255 in FY2003 to almost 2000 in FY2012.

September     Recently, NPR (through the work of Richard Harris and colleagues) aired a series of 7 stories about biomedical research and NIH funding with 5 stories on Morning Edition

October     I am now starting to analyze the publication patterns of K99-R00 awardees. For this study, I examined the initial 2007 K99 cohort of 182 investigators, of whom 170 transitioned to R00 awards

November     In my initial post on this program, I noted that recipients of F32 awards during the same period might make a reasonable group to compare to the K99-R00 recipients. I have now performed some of this analysis.

December     I filed a request to the NIH through the Freedom of Information Act (FOIA) on October 12 requesting available age data for New Investigators and ESIs for fiscal years 2006 through the present. Today, I received an email with an attached spreadsheet responsive to my request.

 

7 responses so far

Estimated New Investigator Distribution for FY2014 by Institute and Center

(by datahound) Dec 03 2014

It dawned on me that with the list of competing R01 grants from 2001 to 2014, I could identify investigators who received competing R01 grants in FY2014, but who had not received any such grant from FY2001-FY2013. This represents a reasonable method for identifying New Investigators although it will over-count those who had received substantial non-Ro1 support from NIH previously. Using this approach, I identified 1352 "New Investigators" out of a total of 5001 total competing R01s. This number of "New Investigators" is consistent with the expected number based on the recent data that I obtained from NIH.

Of the 5001 grants, 1438 are Type 2 (competing renewal) awards. This leaves 3563 Type 1 (new) awards. Of these 1352 went to "New Investigators" leaving 2211 new awards going to investigators who had previous received an R01 award from NIH.

My list does not distinguish Early Stage Investigators from other New Investigators. Further work will be required to try to sort this out. Nonetheless, I can examine factors such as the distributions across NIH Institutes and Centers, institutions, etc.

The distribution across NIH Institutes and Centers is shown below:

IC Total R01s New PI New/Total
NIGMS 817 187 0.229
NCI 628 176 0.280
NIDDK 472 128 0.271
NIAID 443 121 0.273
NHLBI 514 121 0.235
NINDS 370 100 0.270
NIMH 266 88 0.331
NICHD 171 67 0.392
NIA 177 59 0.333
NIDA 184 45 0.245
NEI 230 44 0.191
NIEHS 95 32 0.337
NIBIB 80 31 0.388
NIAMS 133 30 0.226
NIDCD 112 26 0.232
NIDCR 70 24 0.343
NIMHD 34 18 0.529
NINR 35 16 0.457
NIAAA 88 16 0.182
NHGRI 36 10 0.278
NLM 18 8 0.444
NCCAM 21 5 0.238

This list does show some differences between ICs (in terms of the ratio of "New Investigator" awards to total competing Ro1s.

While this approach is based on some assumptions, it should be quite useful for estimating such parameters in a better than "back of the envelope" sense. Suggestions for other parameters to look at are welcome.

4 responses so far

Age Distributions for NIH New Investigators and Early Stage Investigators

(by datahound) Dec 03 2014

One of the most quoted statistics about the NIH is that the average age of an investigator receiving their first R01 is approximately 42. The increasing age of "New Investigators" has been the cause of considerable concern across NIH and the scientific community. When I was at NIH, many realized that the definition of "New Investigator" as someone who had not previously received substantial NIH funding led to a quite heterogeneous group. New Investigators, who many imagined as scientists in the early stages of their careers also included senior scientists who came from other fields (where their research support had come from NSF and other non-NIH agencies) or from other situations (such as from other countries) where their research had been supported by other agencies. NIH did some internal analysis that revealed that approximately half of the "new investigators" were in the early stages of their careers while the other half were more senior. This led to the definition of an "early stage investigator" or ESI as someone who was within 10 years of their terminal degree or the end of their clinical training.

As my readers have likely discovered, I feel that single statistics such as "an average age of 42" is dangerous to interpret without looking at the data and distributions that underlie such figures. After searching online and asking several sources at NIH if data about the age distributions were publicly available without success, I filed a request to the NIH through the Freedom of Information Act (FOIA) on October 12 requesting available age data for New Investigators and ESIs for fiscal years 2006 through the present. Today, I received an email with an attached spreadsheet responsive to my request.

The spreadsheet contains New Investigator data from FY2006 through FY2013 and ESI data from FY2009 through FY2013. The data has a few limitations. First, the age distributions below 31 and below and 55 and above are binned. Second, cells that would contain fewer than 10 are left blank. While this is to protect the anonymity of individuals who would fall in these cells, I am not sure how this applies here. Third, age data are not available for approximately 8-10% of these grantees.

The age distributions for New Investigators are shown below:

New Investigator plot

This plot shows relatively little change from Fy2006 to FY2013. The medians calculated from these data are shown below:

2006 40.4
2007 40.6
2008 40.6
2009 41.1
2010 40.9
2011 41.2
2012 41.3
2013 41.0

These values support the conclusion that little change has occurred.

The age distributions for Early Stage Investigators are shown below:

ESI Plot

These distributions also show relatively little change. The distributions are approximately Gaussian with relatively skew, suggesting that the limit of being within 10 years of the terminal degree is not having a dramatic effort.

The medians for these distributions are shown below:

2009 37.6
2010 37.6
2011 38.4
2012 38.6
2013 38.0

Again, relatively little change has occurred; if anything, the median age appears to have increased slightly over this period.

Finally, the availability of both distributions allows the calculation of the distributions for non-ESI New Investigators. The missing ESI data (due to cells with fewer than 10 grantees) were estimated by fitting Gaussians to the distributions. The curves for ESI and non-ESI New Investigators are compared below:

ESI-NonESI Plot

The curves for non-ESI New Investigators are skewed with one, relatively steep, arm with a halfway point slightly above age 40 and the other more gradual arm with a halfway point near 50. The medians for the non-ESI New Investigators are shown below:

2009 45.4
2010 45.1
2011 46.6
2012 45.9
2013 46.2

The medians here appear to have move up approximately 1 year over this period.

One final parameter of interest is the percentage of New Investigators who are ESIs. NIH had discussed trying to substantial increase this percentage over time. These percentages are shown below:

2009 50.1
2010 46.9
2011 54.5
2012 57.1
2013 54.0

The percentage does appear to have increase to some extent over this period although the increase is relatively modest.

I welcome your thoughts about these data and what they might suggest in terms of the success of current or potential new NIH policies.

20 responses so far

Distribution of R01s Across Institutions and Individuals 2001-2014

(by datahound) Dec 02 2014

In the course of my recent analyses, I generated a data base including 84739 competing (new and competing renewal) R01 grants awarded by NIH from FY2001 to FY2014. These grants were awarded to approximately 1300 institutions although approximately half of these institutions were awarded 3 or fewer R01s over this period. The total number of unique principal investigators is approximately 44200. The distribution of the number of grants and the associated number of principal investigators for institutions with 10 or more R01s over this period is shown below:

Institutions-PI distrib plot

 

The two curves are approximately parallel, indicating a relatively constant ratio of competing R01 grant awards over this period per principal investigator. Overall, this average is 1.91.

The distribution of the number of principal investigators versus the number of competing R01 grants awards from FY2001 to FY2014 is shown below:

 

Grants per PI Institutional Dist Plot

 

The number of awards ranges from 1 (since only investigators with at least 1 R01 award are including in the analysis) to 16. This curve appears to be exponential as is supported by the plot of the natural log of the number of principal investigators versus the number of competing grants shown below:

Ln Number of Institutions vs Number Grants

This plot is quite linear with a slope of -0.675 (corresponding to the exponent in the initial curve).

This distribution of ratio of competing R01 grants per principal investigator across 150 institutions with a relatively large number of awards is shown below:

R01 per PI histogram

The 25 institutions with the highest value of this parameter are listed below:

JOSLIN DIABETES CENTER 3.37
SCRIPPS RESEARCH INSTITUTE 2.96
WISTAR INSTITUTE 2.95
ROCKEFELLER UNIVERSITY 2.92
LA JOLLA INST FOR ALLERGY & IMMUNOLGY 2.83
SALK INSTITUTE FOR BIOLOGICAL STUDIES 2.81
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 2.69
HARVARD UNIVERSITY (MEDICAL SCHOOL) 2.61
SANFORD-BURNHAM MEDICAL RESEARCH INSTIT 2.60
CALIFORNIA INSTITUTE OF TECHNOLOGY 2.59
DANA-FARBER CANCER INST 2.53
UNIVERSITY OF CALIFORNIA BERKELEY 2.52
STANFORD UNIVERSITY 2.52
JACKSON LABORATORY 2.51
HENRY M. JACKSON FDN FOR THE ADV MIL/MED 2.51
BRANDEIS UNIVERSITY 2.36
UNIVERSITY OF PENNSYLVANIA 2.33
COLUMBIA UNIV NEW YORK MORNINGSIDE 2.32
J. DAVID GLADSTONE INSTITUTES 2.32
UNIVERSITY OF COLORADO 2.31
WASHINGTON UNIVERSITY 2.29
UNIV OF MASSACHUSETTS MED SCH WORCESTER 2.29
FRED HUTCHINSON CAN RES CTR 2.29
YALE UNIVERSITY 2.26
WEILL MEDICAL COLL OF CORNELL UNIV 2.25

This list includes groups of institutions with very different "business models". Some operate largely on "soft money" where principal investigators are expected to bring in most of their salaries (as well as research support) from extramural sources. Others are largely basic science-focused institutions with significant institutional support through teaching and other missions.

How do the curves of the number of principal investigators versus the number of R01 grants look for these institutions?

The curves for Joslin Diabetes Center are shown below:

Joslin plot-2

These curves are relatively noisy since it is based on only 27 principal investigators. With this caveat, the fit reveals an exponent of -0.209, smaller by more than a factor of three than the overall NIH-wide parameter. This reveals that a larger percentage of principal investigators have a greater number of R01s over this period of time.

The curves for Scripps Research Institute, Rockefeller University, and MIT are shown below:

Scripps plot-2

Rockefeller Plot-2

MIT Plot-2

Comparison of these graphs reveals that each distribution is approximately exponential although some variations are present. The calculated exponents are -0.367 for Scripps Research Institute, -0.381 for Rockefeller University, and -0.435 for MIT. These track the ratios of the total number of competing R01s to principal investigators.

As a final point of comparison, the graphs for Massachusetts General Hospital which has a ratio of competing R01s to principal investigators of 2.01, slightly above the NIH-wide value.

MGH Plot-2

Again, an approximately exponential distribution is observed with a coefficient of -0.578, relatively close to the NIH-wide value.

This analysis reveals that the distribution of the number of competing R01 grants received by investigators over the period from FY2001 to FY2014 is remarkably constant across a range of institutions. The exponential distributions indicate that the experiences of principal investigators vary widely across each institution with a small number of investigators highly successful in obtaining R01 funding and a larger number of investigators obtaining smaller numbers of competing R01 awards over this 14 year period.

3 responses so far

More on Institutional Distributions of the Fraction of A0 Applications within the Pool of Funded R01 Grants

(by datahound) Dec 01 2014

In my previous post, I presented (among other things) the distribution of the percentage of A0 applications within the pool of funded  R01 grants across institutions. This distribution showed that, among the top 100 institutions in terms of total NIH funding in fiscal year 2013, some institutions showed notably higher fractions of A0 applications among their funded R01 grants. Based on this and other analyses, this suggested that this supports the notion that these institutions enjoy higher success rates for their R01 applications. This surrogate may be useful since NIH does not release success rates by institution.

Before I present further analyses of these data, I need to note some technical issues. The data present in the NIH RePORTER data base can be challenging with regard to longitudinal analysis across institutions. While the data are relatively clean with regard the use of institutional names within a given year, they are less clean moving from one year to the next for many institutions. Institutional names can vary through the use of different abbreviations, punctuation, and so one. For example, the University of Michigan is listed as UNIVERSITY OF MICHIGAN AT ANN ARBOR for 2001-2006 and UNIVERSITY OF MICHIGAN for 2007-2014. This required construction of a data set with these ambiguities removed as much as possible. In addition, in my first analysis I determined the percentage of funded A0 applications among R01 grants for each year and then averaged these values. A more robust approach is to determine the total number of funded A0 applications for all years and then divide this value by the total number of R01 grants over the same period. I will use this method for all subsequent analyses.

In considering why some institutions show a higher percentage of A0 applications among their funded R01 grants, one important fact to keep in mind is that the success rates for competing renewal (Type 2) applications are generally higher (typically by approximately a factor of 2) than those for new (Type 1) applications. For example, in FY2013, the overall R01 success rate across NIH was 17%. This breaks down as a success rate of 14% for new (Type 1) applications and 31% for competing renewal (Type 2) applications. This observation suggests two possible explanations for the higher overall percentage of A0 applications among R01 grants at some institutions. First, the percentage of competing renewal (Type 2) grants among the funded R01s might be larger for some institutions than for others and this might account for the higher percentage of A0 applications overall in the grant pool. Second, the percentage of A0 applications among R01 grants might be higher even for new (Type 1) applications for these institutions. Of course, these explanations are not mutually exclusive. They are essentially independent and may or may not be correlated by virtue of institutional characteristics.

The distribution of funded Type 2 applications among funded R01 grants across institutions are shown below:

Fraction T2 histogram

The institutions with the higher fraction of Type 2 applications are:

1 MASSACHUSETTS INSTITUTE OF TECHNOLOGY
2 JOSLIN DIABETES CENTER
3 ROCKEFELLER UNIVERSITY
4 BRANDEIS UNIVERSITY
5 UNIVERSITY OF CALIFORNIA BERKELEY
6 CORNELL UNIVERSITY
7 INDIANA UNIVERSITY BLOOMINGTON
8 UNIVERSITY OF OREGON
9 PRINCETON UNIVERSITY
10 STATE UNIVERSITY NEW YORK STONY BROOK
11 JACKSON LABORATORY
12 HARVARD UNIVERSITY (MEDICAL SCHOOL)
13 UNIVERSITY OF CALIFORNIA SANTA CRUZ
14 HARVARD UNIVERSITY
15 SCRIPPS RESEARCH INSTITUTE
16 TUFTS UNIVERSITY BOSTON
17 NEW YORK UNIVERSITY
18 COLORADO STATE UNIVERSITY
19 UNIVERSITY OF WISCONSIN-MADISON
20 OREGON HEALTH & SCIENCE UNIVERSITY
21 UNIVERSITY OF COLORADO
22 PENNSYLVANIA STATE UNIVERSITY
23 UNIVERSITY OF ILLINOIS URBANA-CHAMPAIGN
24 CALIFORNIA INSTITUTE OF TECHNOLOGY
25 RUTGERS

At the other end of the distribution, institutions with relatively low fractions of Type 2 applications in their funded R01 pools include a number of major clinical centers including Seattle Children's Hospital (0.167), Cincinnati Children's Hospital (0.225), M.D. Anderson Cancer Center (0.268), Massachusetts General Hospital (0.285), and Brigham and Women's Hospital (0.297). This may reflect the lower rate of submission of Type 2 applications for clinical (as opposed to basic science) studies noted in an NIH publication from 2008.

The distribution of the percentage of A0 applications within the pool of R01 grants across institutions is shown below:

Type 1 Percent A0 histogram

The institutions with the higher percentages of A0 applications among their funded R01 grants are:

1 ROCKEFELLER UNIVERSITY
2 PRINCETON UNIVERSITY
3 CALIFORNIA INSTITUTE OF TECHNOLOGY
4 UNIVERSITY OF CALIFORNIA SANTA CRUZ
5 SALK INSTITUTE FOR BIOLOGICAL STUDIES
6 J. DAVID GLADSTONE INSTITUTES
7 MASSACHUSETTS INSTITUTE OF TECHNOLOGY
8 COLD SPRING HARBOR LABORATORY
9 HARVARD UNIVERSITY
10 CORNELL UNIVERSITY
11 UNIVERSITY OF CALIFORNIA BERKELEY
12 MOREHOUSE SCHOOL OF MEDICINE
13 FRED HUTCHINSON CAN RES CTR
14 DANA-FARBER CANCER INST
15 CHILDREN'S HOSPITAL CORPORATION
16 COLUMBIA UNIV NEW YORK MORNINGSIDE
17 UNIVERSITY OF COLORADO
18 INDIANA UNIVERSITY BLOOMINGTON
19 STANFORD UNIVERSITY
20 HARVARD UNIVERSITY (MEDICAL SCHOOL)
21 RUTGERS THE ST UNIV OF NJ NEW BRUNSWICK
22 OREGON HEALTH & SCIENCE UNIVERSITY
23 UNIVERSITY OF CHICAGO
24 CINCINNATI CHILDRENS HOSP MED CTR
25 BROWN UNIVERSITY

Note that 10 institutions appear on both lists. Indeed, these two parameters are substantially correlated as shown below:

T1-Percent A0 vs Frac T2 plot

The correlation coefficient between these two parameters is approximately 0.4. These data reveal that both factors contribute to the increased percentage of A0 applications among funded R01 grants at some institutions.

These analyses provide some data that may help sort out the factors that contribute to the higher percentage of funded A0 applications among R01 grants at some institutions, including those factors that contribute to the success of applications such as the reputations and seniority of the applicants, institutional biases in peer review, and other factors. I welcome comments about how these analyses might be extended.

6 responses so far

Percentages of A0 Applications in the Funded Pool by Institute and by Institution

(by datahound) Nov 25 2014

In recent posts, I presented data regarding the trends in the percentage of A0 applications among funded R01 grants, showed how the percentage of A0 applications varied between the NIH-wide population and a select group (members of a section of the National Academy of Sciences), and examined the variations in R01 success rates between different NIH Institutes and Centers. Here, I bring these threads together to look at how the percentage of A0 applications among R01 grants varies across NIH Institutes and Centers and then extend the analysis to see how these percentages varies across different universities and other institutions.

Curves showing the percentages of A0 applications among funded R01 grants for six NIH institutes are shown below:

6 IC percent A0 curves

 

The top two curves are from NIGMS and NEI, two institutes that have had relatively high R01 success rates. The curves lie above the NIH-wide curve. The other curves are for four large institutes with lower success rates.

The average percentages of A0 applications among funded R01 grants over the period from FY2001 to FY2013 for these institutes are compared with the average R01 success rates over this period are shown below:

Institute Average % A0s Ave Success Rate
NIGMS 0.556 0.290
NEI 0.533 0.325
NCI 0.380 0.205
NIAID 0.399 0.232
NHLBI 0.457 0.235
NICHD 0.389 0.175

These parameters are highly correlated with a correlation coefficient of 0.90. The slope of the line fit to these data is 0.54 ± 0.09. Thus, s a rule of thumb, the average success rate is approximately one half of the average percentage of A0 applications among funded R01 grants over this period.

These data are extended to all ICs with R01 success rates for individual fiscal years plotted versus he percentages of A0 applications among funded R01s for those years below:

Correlation A0 percent-SR

 

As could be anticipated, there is more scatter in these data, given the factors that influence both parameters as well as the changes in policy over this period. Nonetheless, a correlation is observed with a correlation coefficient of approximately 0.4 and a best fit line with a similar slope.

The observation of this correlation suggested that the percentage of A0 applications among funded R01 grants could serve as a publicly available parameter to examine the experiences of average investigators at different extramural institutions. The average percentages of A0 applications among funded R01 grants for the period over the period from FY2001 to FY2014 were calculated for the 100 institutions that received the most NIH funding in FY2013. This distribution of these averages are shown below:

100 Institution histogram-2

The distribution includes a number of institutions with relatively high percentages of A0 applications. These are labelled above. Examination reveals that these institutions consist largely of basic science-focused institutions with substantial hard money support for salaries including the prominent schools of Arts and Sciences or Engineering. This observation is supported by the fact that Princeton University (which is not in the top 100 institutions in terms of overall NIH funding) has a high percentage of A0 applications (62.2%).

Curves for individual institutions are compared below. These include the institution with higher percentage (Rockefeller), an institution with one of the highest percentage but not an outlier (UCSF), and application in the center of the distribution (University of Pennsylvania), and an institution at the bottom edge of the distribution (Wayne State University).

Percent A0 Institutions-2

Based on the correlation with success rates, these data suggest that the average success rate for investigators at Rockefeller is approximately 50% higher than that for the NIH-wide average.

16 responses so far

R01 and Other Mechanism Funding Trends 1998-2013

(by datahound) Nov 20 2014

In my most recent post, I noted that the percentage of the overall NIH budget going to R01-equivalent awards dropped from 43.8% in FY1998 to 37.2% in FY2003 to 34.7% in FY2013. Here, I present more details about the trends that have led to this drop. Below is a plot of expenditures in different categories from FY1998 to FY2013.

1998-2013 All

 

In addition to R01 equivalent grants (R01s, R37s, R29s), the categories include Research Centers, R&D Contracts, Intramural Research, Training (Ts and Fs), Career Awards, Research Project Grants (RPGs) other than R01-equivalent grants, SBIR/STTR grants, Other Research (e.g. R25s), and Research Management and Supports (e.g. extramural staff salaries, other administrative costs, review costs).

The categories other that R01 equivalent grants are shown with an expanded scale below:

1998-2013 Blowup

The graphs show that the percentage of R01 equivalent fell in two different phases. The first phase occurred during the NIH doubling whereas the second phase occurred over the subsequent decade.

The percentages and differences in the first and second phases in the various categories are tabulated below:

Category FY1998 FY2003 FY2013 2003-1998 2013-2003
R01 equivalent 43.8 37.2 34.7 -6.6 -2.5
Centers 8.6 9.2 8.3 0.6 -0.9
R&D Contracts 5.8 8.5 10.0 2.7 1.5
Intramural 10.5 9.4 11.1 -1.1 1.7
Training 3.1 2.7 2.5 -0.4 -0.2
Career 1.7 2.1 2.1 0.4 0.0
RPG Non-R01 equiv 11.2 14.4 15.9 3.2 1.5
SBIR/STTR 2.0 2.0 2.2 0.0 0.2
Other Research 4.6 6.0 6.2 1.4 0.2
RMS 3.6 3.4 5.1 -0.2 1.7

Note that the categories do not total 100% as some small categories such as construction are not shown.

During the first phase, the percentage of the overall NIH budget going to R01 equivalent awards fell by 6.6%. Over this period, the amount going to other categories of RPGs grew by 3.2% (to be discussed below), the percentage going to R&D Contracts grew by 2.7%, and the percentage going to Other Research grew by 1.4%.

During the second phase, the percentage going to R01 equivalent awards fell by an additional 2.5%. This was associated with an additional increase of 1.5% going to R%D contracts, an increase of 1.5% going to non-R01 RPGs, and increase of 1.7% going to Intramural Research (part of which is due to an accounting change), and a 1.5% increase in Research Management and Support.

The activity codes that make up the increase in non-R01 equivalent RPGs are shown below:

Other RPG mechs

Over this period, the largest contributors to the increase are cooperative agreements (which have significant NIH staff involvement), primarily U01s but also U19s and the growth in R21s (driven by a large increase in the number of applications). The investment in Program Project grants (P01s) actually fell over this period. The addition of new mechanisms including NIH Director's Pioneer (DP1) and New Innovator (DP2) awards as well as R00 made a small contribution to the growth.

These data are aggregated across all of NIH. As the previous post showed, there are significant differences between ICs. Further analysis will be required to analyze these trends.

4 responses so far

R01 Success Rates by Institute and Center

(by datahound) Nov 20 2014

Each NIH Institute and Center (IC) has its own leadership and, to some extent, policies, practices, and priorities. Furthermore, each receives a separate appropriation from Congress. Because of these influences, factors of considerable interest to the scientific community vary from one IC to another.

One such factor is the success rate for R01 applications. Note that "success rate" is related to but different from "payline". The success rate is the number of awards made in a given fiscal year divided by the number of applications reviewed in that year. The number of applications reviewed includes both scored and unscored applications but applications for the same grant (e.g. an A0 and an A1) reviewed in the same year are only counted once. The payline is a percentile threshold that some ICs use in a manner that almost all applications that score better than the payline are funded. This difference has been the source of much confusion over the years. Rock TalkNIGMS, NIAID, and likely other ICs) have posted explanations.

Success rate data are available through NIH RePORT. The data for FY 2014 (which ended September 30, 2014) are not yet available. For FY 2013, the results for R01 success rates are as follows:

NIH Overall 17%

FIC 0.19
NCCAM 0.12
NCATS N/A
NCI 0.15
NEI 0.28
NHGRI 0.28
NHLBI 0.16
NIA 0.13
NIAAA 0.21
NIAID 0.15
NIAMS 0.17
NIBIB 0.17
NICHD 0.12
NIDA 0.19
NIDCD 0.26
NIDCR 0.22
NIDDK 0.18
NIEHS 0.15
NIGMS 0.21
NIMH 0.19
NIMHD 0.04
NINDS 0.20
NINR 0.12
NLM 0.16

 

Note that these range from 4% to 28%. Some ICs have rather specialized missions and do not accept many R01 applications. These include NCCAM, NCATS, NHGRI, NIMHD, NINR, and NLM. These will be excluded from the rest of this analysis.

What characteristics of the ICs correlate with R01 success rate? One possibility is the overall size of the IC budget. Success rates are shown below with the ICs ordered in terms of budget size.

R01 SR versus Budget

 

There is, in fact, a negative correlation (correlation coefficient -.39) such that larger ICs tend to have lower R01 success rates.

An alternative is the IC investment (in $) in R01s. Success rates are plotted below with the ICs ordered in terms of the size of the R01 investment.

R01 SR vs R01 investment

While there is an increasing trend for the ICs with a smaller R01 investment, the overall correlation is again negative (correlation coefficient -0.23).

A final characteristic is the percentage of the IC budget committed to R01 funding. These percentages ranged from 23.6% to 58.3% in FY2013.

A plot versus these percentages is shown below:

R01 SR vs Percent RO1

Here, a substantial positive correlation is observed with a correlation coefficient of 0.70.

Note that the some of the larger ICs with relatively low percentages of their budgets invested in R01s have considerable other responsibilities for infrastructure or specialized programs. Nonetheless, these data reveal one potential degree of freedom that could help mitigate the historically low R01 success rates that we are now experiencing.

Note that the overall investment in R01 equivalent grants (as a percentage of the NIH budget) has declined over the years. At the start of the doubling (FY1998), this percentage was 43.8%. At the end of the doubling (FY2003), this had declined to 37.2%. In FY2013, it stood at 34.7%.

 

10 responses so far

Older posts »