The Tension Between Average Grant Size and Success Rate

Feb 01 2015 Published by under Uncategorized

Once of the most challenging and contentious issues that is faced by the leadership of NIH institutes and centers involves policies about cutting requested budgets for R01 grants. Once the amount of funds available for new and competing grants is known, the first question is how to divide the pie.

One side of the argument is that making as many awards as possible is crucial since having a funded grant can be the difference between facilitating the launch of a promising career for a new faculty member or shutting down a lab even if it is productive.

The other side of the argument is that requested budget reflect the real costs of the proposed research and that cutting requested budgets substantially can result in systematically underfunded grants for which the budget cannot support the necessary research efforts. This situation may be exacerbated by the modular cap which has remained unchanged since it was created and evidence indicates that PIs may be requesting $250K at the cap rather than requesting a larger budget and running the risk of suggested budget cuts from the study section.

Some policies come into play. First, NIH policy requires that a PI be allowed to modify his/her specific aims if the budget is cut by more than 25%. This plays a role in limiting the size of cuts and also has led to a large number of R01s awarded at the level of $188K (since $250K (modular cap) X .75 =187.5K. Second, Congress often includes language associated with appropriations bills that indicates that, for example, average grant sizes must be the same as in the previous year. This can limit flexibility within NIH institutes and centers.

As the "NIH Doubling" ended, it was easier to make the case that cutting grant sizes a bit while keeping more labs funded made sense since hope sprang eternal that budget increases larger than BRDPI were coming soon. However, as the decade of flat budgets continued, followed by the sequester, this became more problematic. The net results was that the buying power (size increase corrected for BRDPI) of the median R01 grant fell by 18% from 2003 to 2013.

What steps should be taken now? Should the modular cap be increases (or dropped altogether)? Should cuts to grants when they are awarded be minimized? Should cuts to non-competing grants be discontinued? Other suggestions? Bear in mind that the laws of arithmetic still apply and any increase in average grant size will result in a decrease in the number of grants that can be funded.

31 responses so far

  • Drugmonkey says:

    I think that after a decade the NIH should recognize the new normal. This budget cutting is one of the things that amps up the grant churn and it should be a 2-3 yr stopgap at the most. It is time to stop cutting budgets and time to increase the modular to $375k per year.

    Regarding modifying Aims.... Hahahaha, good luck convincing the study section reviewing your competing continuation of that. I have not found this to matter one iota during review. Probably *especially* since it happens to everyone these days.

  • lurker says:

    Thanks, DH, for the updated CDF plot of single R01's dollar amounts vs % of total investigators, but does it seem like the plot is masking or shifted to the right of actually representing what you are writing here about many R01's shaved by a module to $188K?

    I can think of a step to remedy what is becoming the conversion of the NIH into the healthcare version of the Military Industrialized Complex. What we're calling Over-mod grants, which is surprisingly numerous and large, is analogous to massive contracts going to the few giant bloated defense industries making us new $400B F-35 fighter jets. Soon, the barriers to entry for noobs and small town grocer labs will be so high that one has to be a BSD handpicked chosen one to get into or stay in this game.

    Examples of the Biomedical Industrialized Complex are no more clearer than the recent BRAIN-Initiative and now Personalized Medicine Initiative. These dollops of money are only going to the few already entrenched and large groups with distortedly huge influence over the review of such grants. Will the NIH honchos learn what a colossal waste of research efficiency it was from shutting down the Protein Structure Initiative (what? Another Initiative?) , those ridiculous "Roadmaps" programs, and Encode?

    If there is to be salvation, I see there are alot of resources at the top 50% of the feeding trough to shave and more properly distribute, and get a way better return on investment in their research dollars. I doubt though that the NIH would wise up and more properly balancing the portfolio from the entrenched to the new and fresh. In a sense, the NIH needs to become more like the NSF and less like the Pentagon.

    We need to shave all the big Extramural projects and convert them into Intramural programs (genome projects, massive drug compound screens, GWAS), dispense with the intramural smaller labs that would be more appropriate and better served in the extramural where they would be wasting as much resources twiddling bureaucracy, and return to the original mantra of investigator initiated grants all around.

    I know, my idea is non-realistic and asking for the NIH to start all over. But hey, same could be said about Congress, kick everyone out and start all over.

  • Comradde PhysioProffe says:

    We need to shave all the big Extramural projects and convert them into Intramural programs (genome projects, massive drug compound screens, GWAS), dispense with the intramural smaller labs that would be more appropriate and better served in the extramural where they would be wasting as much resources twiddling bureaucracy, and return to the original mantra of investigator initiated grants all around.

    KILL EVERYONE BUT MEEEEEEEEEEEEEEEE!!!!!

  • anonymous postdoc says:

    A modest proposal : if the choice is between success rates, award size, and maintaining or increasing investigator population, I support the cull. Even if I don't get my own lab. I could deal with the change of plan much more flexibly now than 3-6 years from now. (But I guess for now I'm still trying, though I know many who got out into those medical liaison jobs while the getting was good.) Kill the ESI bonus. Let's acknowledge that this system is eating the young, and do it with gusto!

    If we still have to choose between award size and award number once we are down to 200 boomers, I vote for increasing award size. Fewer labs to employ techs and grad students and postdocs that way.

  • rs says:

    I don't see a reason to increase the award size. In fact, NIH should stop supporting PI salaries beyond a certain percent efforts and universities should take the responsibilities for their faculties. The only support NIH should do in a research proposal should be for professional staff (no post-docs or grad students) and actual research cost. If any lab needs a professional staff, there need to be a serious justification for that. Postdocs and grad students should be supported via training grants thus keeping their numbers under check. Most of the industry jobs do not need a PhD and there is no reason to train them for multiple years. Under this scheme a modular budget is pretty good.

  • E rook says:

    I think peer reviewers at SS should not see the or make recommendations on budgets or justifications. Supposedly they focus on teh meritz, but I've heard of more than one NI/ESI change their proposals (and be advised to do so), to be within modular for R01s, which we can now agree seems arbitrary in the face of inflationary costs to carry out the (SS's) expected breadth and depth of an R01, 3-aim, project. I think that Councils have the expertise to judge the cost of carrying out the research and peer review should be divorced from cost considerations. Supposedly the decision to fund a project at the IC) is separate Lready. From an evaluation of teh scienze, I advocate removing budget considerations from the minds/biases of the indivuals Eva,urged teh meritz. Council and the IC Directors' offices should have the discretion to negotiate adjusting aims to budget cuts. Not SS. I think the groupthink and distance from reality make them unqualified (think Reagan never seeing a bar code scanner at the grocery store and W not knowing the price of a gallon of gas).

  • E rook says:

    Correction I meant daddy bush, not Reagan, but the principle holds.

  • Comradde PhysioProffe says:

    Cutting grant budgets to keep paylines up is a mechanism for imposing de facto involuntary cost sharing on grant recipient institutions.

  • Established PI says:

    The whole concept of modular budgets should be scrapped. I am having trouble reconstructing why the modular cap was ever conceived. I believe the idea was to somehow redirect the study section's attention to the science rather than the budget but there are way better ways to do this. Since they virtually always cut the budget anyway, it's nuts to stick to modular budgets when you can submit a real budget on which you can still do the science after the inevitable cut. I just did this for my more recent new R01 and the study section never commented on the non-modularity. After the cut, I was a bit over the modular cap and am left with enough money to do what I proposed (whew!).

    A far simpler way to keep the attention on the science is for the "budget" to be simply in terms of WHAT the investigator is requesting to do the work: N postdocs, M students, p technians, X% salary, plus a per-person figure for supplies. That is the way we all think about projects and that is what study sections focus on anyway, so why not just put the budget justification in those terms? Then let the professionals inside the NIH worry about actual dollar amounts in the <10% chance the grant is actually considered for funding.

    The bigger question, though, is how much the current modular cap is contributing to grant churn, submission of more applications and low success rates. I suspect you would simply see more spread in your distribution but not much change unless the NIH were to make the hard choice of reducing the number of funded PIs and increase the average award size overall. I think the hyperfocus on the relatively few PIs with very large (or very many) R01s misses the larger point. It's sort of like the question of how to raise tax revenues: there just aren't enough rich people to tax so you have to go after the broad middle class to really raise sufficient revenue. Same with grants: you need to reduce the numbers of PIs in the middle applying for grants if you really want to raise success rates as well as the average size of grants. Killing big programs and limited total $$s helps some but not enough. I fear that sucking more ESIs into the system only to bump them off later is not the answer.

  • Anne Carpenter says:

    Thanks for this discussion. I firmly believe that spreading the wealth (as thinly as is the case now) is a terrible idea, not good for science nor for scientists. Bravo to the postdoc above calling for a culling: I agree most (myself included) would rather not be in science than be in science under circumstances where funding is so tight.

    Given that, there are two approaches if indeed you want to fund fewer labs but at decent, efficient, sustainable levels. You can (a) increase the size of individual grants (in this discussion, raise the modular cap) and expect labs to mostly have one, or maybe two. And (b) you can actually decrease the size of grants and expect labs to have several. Like, each grant is $100k and a lab has between 2-10 of the little suckers, depending on the context. Seems like the modular cap is heading towards the functional equivalent of approach (b), and I'm not sure it's a terrible thing.

    The downside of (a) is that it leaves a lab very prone to funding gaps (or funding disasters). Most labs cannot ride out the complete loss of grant funding and the funding cycles are too unpredictable. Perhaps if competing continuations were funded 85%+ of the time on the first submission, this would be a good approach, but statistics being what they are, even after a culling this would be a more-stressful-than-necessary process.

    I don't see (b) as having horrendous downsides. One could argue it increases grant churn (and thus writing/reviewing time), and maybe that is true, but I'd suggest that this may happen anyway. In case (a) people will make multiple submissions to guard against losing a lab's entire funding, so each lab would try to have a couple of irons in the fire. (b) would be kind of cool, sort of like applying for fellowship-sized projects, roughly one per person in your lab. It would require a shift in reviewers' thinking in order to assess the outcome of a half dozen projects.

    What seems the most sensible to me is the "people, not projects" approach of MIRA. I'm not certain it is the case, but it seems the mechanism has built into it the idea that at the end of the 5 year period, reviewers will suggest either increasing funding (based on strong performance) or decreasing it (based on so-so performance) but it will be unusual for it to be completely not renewed.

  • drugmonkey says:

    b) has gotten worse and is unnecessary. Your multi-tiny-awards is out of step with current NIH extramural reality, as evidenced by more apps growing to cap and then more apps exceeding cap over the past 10-15 years.

  • drugmonkey says:

    E PI:

    No, we don't "all" budget based on a per-person for supplies *at all*. Jesus, you people. It is LOOOONG past time for you to grasp the diversity of science supported by the NIH and the diversity conducted within a two building radius of your own.

    I do my budget based on the experiments I propose and the costs for doing them. One of those costs is staffing, sure. But I don't propose "I want three postdocs and enough money to keep them busy" like you do. My work isn't like that. But I have friends who do work more like this and I am able to grasp the whys and wherefores with little trouble. It ain't rocketbrainsurgery.

  • Pinko Punko says:

    It is a sort of impossible question. I would like some recognition of the fact that under a certain amount, it is unlikely for a grant to be competitive in terms of what can be accomplished. Once below this threshold it is very difficult. I understand why budgets are cut to keep pay lines up, but cutting across board including budgets that are at bare minimum is regressive. I think these are some of the drivers for the considerations of large grants or multiple grants as being the next target- it is likely because you can't cut the smaller grants any smaller. In essence- those grants are cutting into bone, so there must be some flesh- be it muscle or fat somewhere else. This is all very depressing. The fact that it is very hard to imagine pay lines moving or grant sizes increasing together, but the budget increase that NIH needs to accomplish that is on the order of 10%-15%? If NIH increases in health, the only way to prevent a bubble is for grant sizes to move up at greater rate than the pay lines.

  • Established PI says:

    DM - Yes, I excluded lots of things from my post like mouse colonies, subcontracts, computational projects, lab equipment, machine shop costs, as just a few examples (and that's just for non-clinical research). The general principle holds, namely that the investigator would enumerate principle cost categories needed to carry out the research. This simplifies the budget question and is really all, I would argue, that the study section should consider. Just-in-time information requests would be the time for detailed budgets, which could then be handled and decided upon administratively.

  • Dave says:

    Limiting % salary paid from NIH grants would be a step in the right direction. At the moment, the NIH is paying PIs up to 100% salary, just for them to sit down and write more grants (even though the NIH hides this with the old 98% effort thingy). This will accelerate the cull further.

    The fact that it is very hard to imagine pay lines moving or grant sizes increasing together, but the budget increase that NIH needs to accomplish that is on the order of 10%-15%?

    10-15% seems on the low side for both. The 'success rate' didn't really budge from 98-03 (where the NIH doubling really took hold), because during this period there was an almost 100% increase in the number of applications, which increased linearly until around 2007. It's obvious that the number of applications (read: investigators) will have to decline significantly before things can really improve for those that are left. There is no escaping that.

  • DJMH says:

    Keep modular cap where it is, slightly reduce success rates and/or trim from large program mechs, and use ALL of the saved funds for F31s and 32s. That has many benefits:
    1. If most R01 money goes towards trainee salaries, this frees up some funds, as more trainees will have grants.
    2. Trainees will find it a lot easier to escape awful labs, if they can bring their funds with them, so people will be less pressured to join Famous-Funded-But-Very-Mean Labs.
    3. IDC rate of 8% on F mechs means that this shift in general saves money, at least from NIH's standpoint.
    4. Trainees on F mechs are actively tracked by NIH, so for a change we might have a better idea what happens to most trainees...it's disgraceful that we don't.

    Yes, this makes life a bit harder on labs with a ton of foreign postdocs, unless eligibility rules are changed. But that seems like a worthwhile trade.

  • DJMH says:

    Oh and
    5. Trainees will get practice in grant writing, which should help sort out for a lot of people if this is how they want to spend their lives or not.

  • drugmonkey says:

    I agree with you in principle DJMH, the trouble is that it is going to take more than the sort of minor shaving that you seem to envision.

  • DJMH says:

    Going on 2012 numbers from table presented here, the average RPG (total costs) is $420,000. The average individual training grant costs were $42,599. Therefore, for every one RPG cut, you could fund 10 students or postdocs on training grants.

    From your graph, there are 10,000 postdocs on federal rpgs (let's ignore the ones funded by other mechs). So if we wanted to have funds for all of them to have F32s, that would entail cutting 1000 RPGs, out of a total of 34,000. That would be hard, I agree, but if everyone's postdocs were on F32s, then most people should be able to run their lab on one R01.

    You can rejigger these numbers so that, for example, 5000 postdocs and 5000 grad students are funded, or whatever.

    Obviously, I would also support a solution in which the $3.2 Billion budget for intramural was cut by 13%, leaving regular RPGs intact, which would free up the same amount of money, but I was trying to avoid the "do it to Julia" issue.

  • drugmonkey says:

    You are definitely on to something good here.

  • Established PI says:

    DJMH - your proposal is in line with recent recommendations by Alberts et al and the NAS on funding students and postdocs with training grants or fellowships.

    If you buy into the other recommendations of the NAS report on postdoctoral fellows (http://www.nap.edu/catalog/18982/the-postdoctoral-experience-revisited) , you can make this work even better by cutting the number of postdocs in your calculations in half. However, the other part of that report's recommendations is to increase the number of staff scientist positions. Those would be more expensive per person than postdocs, although the idea is that they would be more productive and hence one would need fewer.

  • SaG says:

    DJMH ,
    Would you limit the number of F32 post-docs any one PI could sponsor? I am just imagining the Post-Doc population of Boston exploding.

    SaG

  • DJMH says:

    I dunno SaG. In general unless there's a strong reason FOR a regulation, it is probably better not to have it. I think there's a limit to how many postdocs a PI can effectively mentor, but it's up to the postdocs to figure that out when they choose labs, not for the NIH to decide from above. I doubt everyone would end up in Boston...PIs will still have to have resources for each trainee (bench space, reagents, monkeys, whatever) so they aren't going to be able to accept everyone.

    EPI, yes, and they're not the first to mention this either.

    The problem with my plan is that it shifts power from the powers-that-be to the powers-that-aren't, and as such would be hard to implement, because power doesn't give up power. Until there's a run on pitchforks.

  • datahound says:

    Thanks to all for an interesting discussion. Keep it going.

    With regard to the issue of moving folks from research grants to fellowships, I am not sure that NIH has the authority to prevent grantees from paying for postdocs on research grants. The same applies to graduate students, although they could make tuition not an allowable expense. If this is correct, how does this influence the options?

  • DJMH says:

    I don't think there's any reason for NIH to prohibit that--and in any case, foreign nationals would still need to be on research grants. But, a lab that runs with two grad students and three postdocs, say, would have previously needed two R01s; and now if the two grad students and two of the three postdocs have F grants, the lab could hypothetically run on one R01. And, the "missing" R01 would also be funding six more trainees in some other labs.

    I also don't think there would need to be an expectation that all students and postdocs have F grants; it would still help a lot, numbers-wise, if half of them did.

  • Established PI says:

    DH - I don't see how the NIH could ever completely ban paying students and postdocs from grants. As many have pointed out, there will be a giant sucking sound as students and postdocs with their own fellowships flock to the Boston and Bay areas. Labs in less attractive locations will be left to fend for themselves and will need the option of paying students/postdocs from grants. The NIH can play a role in ensuring better training by requiring career development (and documentation, sigh) and by strictly limiting the number of years that someone can be paid as a postdoc (cumulative, over all post-Ph.D. positions).

  • Drugmonkey says:

    I for one am sick to death of anyone hiding behind statutory authority skirts when all they have to do is ask Congress for a tweak.

  • Drugmonkey says:

    But DJMH, please. Without a stick your scheme doesn't work. At all.

  • DJMH says:

    Well, I suppose the "stick" would be that NIH would be more skeptical of 2+ R01 labs, the way that supposedly anyone with $1M or more in costs is getting extra scrutiny now. But do you really think it's that dire? After all, it's envisioned as only about a 3% cut to existing RPGs.

    I assume lots of faculty would be *happy* not to be sweating it out for a second R01, whereas I guess you assume everyone will still chase the money as hard as they can?

  • drugmonkey says:

    It is Tragedy of the Commons territory. Enough will still be motivated to maximize their grants that it will pull down the good acts of the more restrained.

Leave a Reply