IC Distributions for R01s from PIs with Multiple R01s

Jun 08 2015 Published by under Uncategorized

In my previous post, I examined the fraction of NIH PIs who had either a single R01 (or R37 Merit Award) or multiple R01s for fiscal year 2014. Overall, about 30% of R01 PIs had more than 1 R01. In the comments and on Twitter, the issue came up about whether those with multiple R01s had them from the same IC or from multiple institutes.

To address this question, I asked the question: If an PI had an R01 from one institute, what is the distribution of ICs for the additional R01s going to the same PI. The results are tabulated below:

IC_pairings_results

NIH Abbreviation Key:  AA=NIAAA, AG =NIA, AI = NIAID, AR = NIAMS, AT=NCCAM, CA=NCI, DA=NIDA, DC=NIDCD, DE=NIDCR, DK=NIDDK, EB=NIBIB, ES=NIEHS, EY=NEI, GM=NIGMS, HD=NICHD, HG=NHGRI, HL=NHLBI, LM=NLM, MD=NIMHD, MH=NIMH, NR=NINR, NS=NINDS

Overall, the percentage of those additional R01s coming from the same IC ranges from 47 to 75%. For those that do not come from the same IC, the number of ICs contributing substantially ranges from a few to many illustrated below (which depicts the data above displayed as the fraction of the R01s from the different ICs given an R01 from a particular IC).

Mult PI IC Graph

For example, if a PI has one Ro1 from AA (NIAAA), 61% of additional R01s come from AA and 18% come from DA (NIDA), leaving 21% for the remaining ICs. In contrast, if a PI has a grant from GM (NIGMS) or CA (NIH), it takes 4 additional ICs to reach 18% of additional R01s.

Which ICs are linked by having PIs with multiple R01s? I examined the top two contributions of additional R01s for each IC (in addition to the IC itself). In these "top two lists", I joined the pairs of ICs. I used a bold line if the link was bi-direcctional, that is, each PI appeared on the top two list of the other. The results are depicted below:

IC-IC graph-2-rev

 

Overall, the patterns that emerge are as might be anticipated. The bidirectional links are between AA-DA, MH-NS, DK-HL, CA-GM, and CA-AI. Some of the larger ICs are linked to many other ICs, reflecting both their size and their relatively broad missions.

UPDATE

As noted in the comments, some of these connections could be attributed to the size of the ICs. Thus, NCI appeared to be linked to many other ICs, but this could be due to the large number of R01s awarded by NCI rather than by actual content overlap.

To address this, I simulated results assuming that the probabilities for an additional grant coming from a particular IC was proportional to the number of grants that this IC award in this data set. I then compared the simulated results with the actual results. Of course, the number of grants going to the same IC was much higher than would be expected. Since this distorted the other statistics, I set all of these values equal to 0 and re-simulated the data. I (or, more correctly, R) performed 1000 simulations and then calculated mean, standard deviation, and other statistics for these distributions of grant numbers. I then compared these with the actual values observed in the data. The results (presented a log(base 10) of the probability of occurring by chance are presented below:

Analyze_Results_2014

 

These results allow assessment of the strength of the interactions corrected for IC size.

The strongest interactions are between NIDA and NIAAA with probabilities of occurring by change of < 10^-88.

The other strong interactions are:

NIMH and NINDS

NIAMS and NIDCR (which was still detected previously even though these are both relatively small ICs)

NIDA and NIMH

NIDDK and NHLBI

NIGMS and NIAID

NIDCD and NEI (which was not detected previously)

The link between NCI and NIGMS is still the strongest link between NCI and another IC, but it is substantially less pronounced that the other links above.

Thanks for the comments. I think this a much improved analysis and I had an excuse to explore additional R tools.

I am now working on generating a 2-dimensional figure that is more consistent with these connectivities in a more formal way.

18 responses so far

  • DJMH says:

    This is very cool! But, I don't understand why the table isn't diagonally symmetric. Is it something where you counted each R01 only once, and "assigned" it to a given PI as their "primary" R01? I think given the type of analysis you're doing, it would make more sense for these to be bidirectionally counted so that the table would be diagonally symmetric.

    • datahound says:

      Yes, I was puzzled about this for a while as well. Consider AT and AA, for example. There is one PI who as 1 R01 from AT and 2 R01s from AA. Thus, if you calculate how many additional grants from AA there are for a PI who has a grant from AT, the answer is 2. On the other hand, if you calculate how many additional grants from AT there are for a PI who has a grant from AA, the answer is 1.

  • drugmonkey says:

    Is there any way to depict the IC links by the respective IC size? Taking NIGMS as an example, of course there will be more links to NCI because of its ginormity. But this doesn't tell us if basic science is being supported equally across the other ICs by the NIGMS.

    Maybe I'm just looking to see your Table presented as percentages and not raw numbers?

    • datahound says:

      The analysis was done based on percentages, not numbers, but IC size certainly does have an effect. For example, I agree that it is not surprising that many NIGMS PIs have additional grants from NCI, it is perhaps more surprising that many NCI grantees have additional grants from NIGMS (although NIGMS is very large in terms of numbers of R01s). I will think further about how to more fully correct for IC sizes.

      • drugmonkey says:

        it is not surprising that many NIGMS PIs have additional grants from NCI, it is perhaps more surprising that many NCI grantees have additional grants from NIGMS

        The question I am trying to get at is whether NCI grantees (or any other IC's grantees) are more likely to have NIGMS funding. I think we can all agree* that if the claims about basic research are valid, then NIGMS should operate in more or less equal support of all the other ICs that we might view as being more disease-focused, translational or whatever your favored term might be.

        *slowgrin

  • chemstructbio says:

    Did you use a plotting program to make the last graph? Or was it done by hand?

    • datahound says:

      By hand. I am working on finding or writing programs that will do this, but I wanted to get something out.

  • SaG says:

    Looking at the numbers "qualitatively" the GM vs. other IC numbers fall roughly in line with the budgets/award numbers of the institutes. CA>AI>HL.....DK and NS numbers are about the same and those ICs have about the same sized budget. The problem is the best number to correct for. CA and AI have large intramural programs so you would have to correct for that for instance. Not to mention the large CFAR and Cancer centers.etc, etc.....

    Have fun with this Datahound. 😉

  • Drugmonkey says:

    That update is so cool. Identifies the closer relationships much better.

    • datahound says:

      Thanks. I was already troubled by the impact of IC size and your question caused me to think about it harder. I agree that the analysis is much more robust and the results are sensible, even down to the relatively modest relationships.

      • drugmonkey says:

        Do you think it appropriate that NIGMS has such clear biases? I would argue that as a very large IC which holds down the "basic science" mandate it should be much more democratic in its relationships across the more disease-focused ICs. It should be a basic science support IC in my view.

        • datahound says:

          The connections between NIGMS and other ICs are around specific programs (NIAID, microbiology; NHGRI, genetics (NHGRI evolved out of an NIGMS program); NCI, signaling; NIEHS, toxicology; NLM, informatics.

          NIGMS is supposed to support research of interest to two or more other ICs or outside the scope of any other IC. I think they do a relatively good job at this. Also, other ICs do (and should) have basic research programs related to their missions.

  • datahound says:

    DM: Please clarify. What do you think is missing from NIGMS' relationships? More connections with other ICs? How should these programs interface with basic science programs in the other ICs?

    • drugmonkey says:

      Your data suggest that NIGMS favors basic research that is relevant to some, but not all, of the other ICs. I am asserting that NIGMS should be egalitarian and support the missions of all of the disease-focused ICs equally. You may disagree, I don't know, but there is a suggestion of a bias here.

      The next step is to move on to the question of why this might might be so.

      Next we can ask if anything *should* be done to redress this.

      Finally we can propose possible solutions.

  • SaG says:

    Is basing NIGMS' relationship to other ICs on the (relatively small) number of PIs who hold 2 or more grants from different ICs an appropriate measure of "bias" for certain research programs?

  • […] institute or center. Furthermore, these data reveal some consequences of this where, for example, similar grants awarded by NINDS and NIMH could be funded at substantially different […]

Leave a Reply