K99-R00 Publication Analysis-Part 1

Oct 19 2014 Published by under Uncategorized

The NIH K99-R00 program is an important program related to the transition from postdoc to faculty positions. This program also presents an unusual opportunity for evaluation since cohorts of scientists at similar career stages compete for initial K99 awards and then can transition to R00 awards and then to R01s and other awards. I have previously posted analysis including the transitions to R00 and R01 grants, gender disparity in R01 transition probabilities, differences between NIH institutes and centers, and gender differences between R0o institutions.

I am now starting to analyze the publication patterns of K99-R00 awardees. For this study, I examined the initial 2007 K99 cohort of 182 investigators, of whom 170 transitioned to R00 awards. I examined the publications of these investigators through the Advanced Search function of PubMed. In many cases, this appeared to produce a relatively comprehensive list of publications based on comparisons with websites and other sources. In other cases, there results appeared problematic due to issues of name ambiguity or a significant number of publications that do not appear in PubMed. Publication lists through the present were generated for 135 investigators.

The total number of publications for each investigator is shown below:

Total Pub Distributions

The number of publications ranges from less than 10 to nearly 100. In some cases for investigators with a relatively small number of publications, technical issues may have resulted in undercounting publications while in a few other cases, the investigators appear to have left academia sometime after receiving the R00 award. Of course, publication numbers have considerable limitations and no attempt has been made at this point to examine individual publications in terms of the citations or other measures.

These publications can be broken down roughly into those leading up to the K99 award and those that occurred after receiving this award. While doing this relatively precisely would require going though individual publications, I used the number of publications in 2007 or before as a surrogate:

Pubs-Pre2007

The publications after 2007 (2008-2014) are shown below:

Post 2007 pubs

These correspond to publications produced during the K99 award, during the R00 award, subsequent publications, as well as some publications of results generated prior to the K99 award that were somewhat slow to be published.

The correlation between the number of publications 2007 and before and the number of post-2007 publication is shown below:

Pre-Post Correlation

Not surprisingly, these are relatively strongly correlated with a correlation coefficient of approximately 0.6. Of course, this reflects differences in the publication patterns between fields and other factors in addition to some more calibrated measure of investigator productivity.

One additional factor that I have examined involves the meme that a publication in Science, Nature, or Cell is highly correlated with receiving a K99 award. Examination of the publication lists reveals that approximately 20% of the K99 awardees have a publication in Science, Nature, Cell or New England Journal of Medicine prior to or in 2007. In addition, approximately 40% have a publication in other relatively high profile journals such as PNAS, other Nature or Cell journals, and the Journal of Clinical Investigation.

With this list of nearly 4000 publications along with the other data that I have assembled on this cohort of investigators, much more analysis is possible and I welcome thoughts about what might be interesting.

23 responses so far

  • drugmonkey says:

    "Only".

    Only 20% have CNS. only 40% have other high-profile. And presumably not all are first-author credits.

    It is important to remember this.

  • datahound says:

    Agreed. I can do more analysis (first author, etc.) but these numbers of not as high as myth would have it. Many have solid publication records in society journals without anything "flashy".

  • Comradde PhysioProffe says:

    I bet it's different in different ICs. I'd be very curious to see this stratified by IC.

    • datahound says:

      Interesting question and good call. I will put up a new post soon.

    • Noncoding Arenay says:

      I too am very curious about this. It will be interesting to see how the factors vary across ICs and will serve as useful information for future applicants aiming for their respective ICs.

  • DJMH says:

    In what field(s) is it common for someone to have 40-60 papers in a 7 year time period as a junior PI?

  • rs says:

    DJMH: when you are a part of big 20+ group and your PI is smart enough to add everyone in every publications, because he knows that it counts in science for everything.

  • EB says:

    I got a lot of advice from the early K99 recipients (much of it through the blogosphere and greatly appreciated) but much had changed when I applied in 2011-12 to NCI. My PO advocated for my borderline application; I was told it was one of 5 in the gray zone. The PO suggested I contact the directors of the Center for Cancer Training and they told me quite directly (by phone) that the weakness in my application was insufficient high impact publications. I had 8 first author papers in society-level journals and a review in Nat Genet, plus a bunch of middle author pubs. First author papers in review could not be considered at all (although I provided manuscripts and letters to show the work was real and substantial). I was told to inform the directors' office right away if I received a manuscript acceptance before the fiscal year deadline. It didn't happen-- the papers were eventually published at Sci Trans Med and a Nature sub-journal-- but not in time for the K99.

  • Noncoding Arenay says:

    @DJMH - This could be computational and might also indicate several productive collaborations. All of them are not corresponding author pubs after all, I suppose.

    If it is not a major time sucker, I would be interested in how this 20%, 40% distribution has changed over the years leading up to the latest round. Presumably, more and more postdocs have been applying as word about the K99/R00 mechanism spreads and this makes me think that the number of applicants with "glamor" or "sub-glamor" pubs might be much higher than the 2007 cohort.

  • DMB says:

    I would like to see how the "high profile publication" pattern looks AFTER K99 phase. Do investigators produce on average the same number of "high profile publications" let's say 5-8 years after transition to R00 phase?

  • lurker says:

    What is going to be the point of this analysis, dear Datahound?

    Anything 1 sigma of the mean, (>20pubs) looks like whacked out citation skimming by a lucky few - are these the lucky R00s going onto the next R01 round? What about the plebians at ~10pubs? How is this cohort fairing (starving)?

    How about just analyze the cohort having between 5 and 15 pubs, and compare it to mid-careers, and then to the graybeards?

    Right now Cell is only considering <40yrs old as upcoming scientists. Maybe your analysis will show that 50 will soon be the new 40 for first time R01?

  • Crystaldoc says:

    I'd be curious to know dates of first senior (last) author publication, and number of senior (last) author publications for each PI, as an indication of how many go on to (attempt to) establish independent research groups, and comparing this to the number that got subsequent R01s. How many senior authorships at time of first R01 award? Perhaps numbers of PIs within the cohort with at least one senior author paper plotted by year, to see if this rises and plateaus, steadily increases (due to some later transitioners) or maybe there will be a peak and tailing off if some are leaving after an attempt at establishing an independent research group but failure to get major grants. How long between activation of R00 phase and first senior authorship?

  • Dave says:

    I don't think you have debunked the 'myth' at all. I think you have confirmed it. Your analysis indicates that 60% of applicants have either a CNS paper, or something very similar (i.e. JCI, PNAS, Nature Neuro, Cell Metabo etc). You have to remember that these are relatively early post-docs and that compared to your garden variety post-doc, these data are truly remarkable. I know very few senior PIs who have these kind of publications also. So one has to be mindful of these comparisons before reaching conclusions.

    • datahound says:

      Dave: The number was somewhat lower than I expected but that may be a reflection of me. There is not doubt that having a high profile publication is very helpful in supporting a K99 application although these data do show that it is not absolutely necessary.

  • drugmonkey says:

    I'd be very curious to see this stratified by IC.

    Me too. I want to see which ICs are wasting so much money on the GlamourJockey chase and PI ego burnishing, instead of getting the science that we pay for published so that real advances in health can be made.

  • Ola says:

    I'd like to see a couple of analyses (although admittedly this would be a LOT of work).

    First, is there an association between publishing glam' (CNS etc) as first author pre-award, and then going on to publish glam' as senior author post-award? Sure, there will be a limited # of superstars who can maintain their glam-pub level science after flying the nest. But, I have a hunch that there are a lot of people coming out of glam labs with 1st author glam-pubs, think they're god's gift to science, and promptly hit the brick wall of reality in which their ability to publish high IF is not as easy as they thought it would be.

    Second (and again difficult), does the publication frequency of the mentor's lab' inform the pattern of the mentee once they leave? I came from a lab where there was a "get it out the door" policy rather than a "save it up for a big one" policy, and this has carried through into my own independent career - publish mid level and frequently. If the pre-K99 pubs were thick and fast, this suggests more about the climate in the mentor's lab'. So, if you could match a single senior author to each pre-award first author, and assume that person is the post-doc' mentor, how do their publication patterns back through history match up with the newly minted trainee running their own lab'. Does glam beget glam? The alternate hypothesis is that these may not be related - the K99-writing post-doc's may just be anomalous superstars among a sea of mediocrity among the postd0c mentors, and they publish a lot because that's what they had to do to "escape", rather than having it drilled into them as trainees.

  • Dave says:

    I would also like to see whether certain mentors (post-doc PIs) have a disproportionate number of K99 awardees. I swear there are some labs that get them over and over and over. Not sure this is a good thing as the future PI gene pool could be significantly smaller.

  • […] response to comments on my recent post, I have examined the IC distribution of both numbers of publications and "high profile" […]

  • […] my first K99-R00 publication analysis post, I presented the distribution of the number of publications for the FY2007 K99-R00 cohort. In this […]

  • […] readers may note the absence of the investigator with the most (95) publications from my initial post on this subject. This investigator only received funding through the R00 phase before moving to a […]

  • […] my first post in this series, I noted that approximately 20% of the 2007 cohort of K99 awardees had a "very high profile" […]

Leave a Reply