Maximizing Investigators' Research Awards for New and Early Stage Investigators: Gender and Race/Ethnicity Issues

(by datahound) Oct 04 2016

In a recent post, I noted that I had submitted a FOIA request for data regarding the NIGMS Maximizing Investigators' Research Awards (MIRA) for New and Early Stage Investigators. My major goal was to get some information about the applications that were administratively rejected. However, I also requested information about the demographics (gender and race/ethnicity) of the applicant and awardee pools. Here, I present an analysis of these data.

First, I need to provide some context. First, the data that I obtained are not quite complete since they were obtained slightly before the end of fiscal year. I do not believe that this will affect any of my conclusions. Second, there is a recent post on the NIGMS Feedback Loop that covers some of these same issues. This post states:

"In addition to ensuring that we are funding the highest quality science across areas associated with NIGMS’ mission, a major goal is to support a broad and diverse portfolio of research topics and investigators. One step in this effort is to make sure that existing skews in the system are not exacerbated during the MIRA selection process. To assess this, we compared the gender, race/ethnicity and age of those MIRA applicants who received an award with those of the applicants who did not receive an award, as well as with New and Early Stage Investigators who received competitive R01 awards in Fiscal Year (FY) 2015.

We did not observe any significant differences in the gender or race/ethnicity distributions of the MIRA grantees as compared to the MIRA applicants who did not receive an award. Both groups were roughly 25% female and included ≤10% of underrepresented racial/ethnic groups. These proportions were also not significantly different from those of the new and early stage R01 grantees. Thus although the MIRA selection process did not yet enhance these aspects of the diversity of the awardee pool relative to the other groups of grantees, it also did not exacerbate the existing skewed distribution."

Let's now turn to the data. With regard to gender, the results are as follows:

Male:  Reviewed applications, not funded: 155, Awards: 63; Total applications reviewed, 218; Administratively rejected applications: 58

Female:  Reviewed applications, not funded: 63, Awards: 19; Total applications reviewed, 82; Administratively rejected applications: 22

Unknown:  Reviewed applications, not funded: 12, Awards: 8; Total applications reviewed, 20; Administratively rejected applications: 3

I will not discuss the "Unknown" category further (gender and race/ethnicity information is provided voluntarily).

From these numbers, we can calculate the following parameters:  Success rate = Awards/Reviewed applications; Probability of administrative rejection = Administratively reject applications/Total applications; All application success rate = Awards/Total applications

Male: Success rate = 28.9%, Probability of administrative rejection = 21.0%, All applications success rate = 22.8%

Female: Success rate = 23.2%, Probability of administrative rejection = 21.1%, All applications success rate = 18.3%

Although these results are not statistically significant, the first two parameters trend in favor of males over females. If these percentages persisted in larger sample sizes, they could become significant.

We now turn to information about the self-identified races of applicants. The categories are: White, Asian, African American, Native American, Multiracial, Unknown, and Withheld. Since the NIH FOIA policy is not to release information for cells that contain 10 or fewer individuals, I did not obtain precise data for African American, Native American, Multiracial, Unknown, or Withheld individuals. Thus, I will present the data as White, Asian, and Other (corresponding to African American, Native American, Multiracial, Unknown, or Withheld). Note that the numbers for the "Other" category can be deduced since the overall total for each category is given.

The data are as follows:

White:  Reviewed applications, not funded: 118, Awards: 63; Total applications reviewed, 181; Administratively rejected applications: 33

Asian:  Reviewed applications, not funded: 71, Awards: 16; Total applications reviewed, 87; Administratively rejected applications: 34

Other:  Reviewed applications, not funded: 41, Awards: 11; Total applications reviewed, 52; Administratively rejected applications: 16

I compiling these totals, I noticed that there are no rows for African American, Native American, or Multiracial in the Awards (Applications Funded) category whereas there are for the other categories. This suggests that there were no awardees who identified as African American, Native American, or Multiracial.

The parameters deduced from these categories are as follows:

White: Success rate = 34.8%, Probability of administrative rejection = 15.4%, All applications success rate = 29.4%

Asian: Success rate = 18.4%, Probability of administrative rejection = 28.1%, All applications success rate = 13.2%

Other: Success rate = 21.1%, Probability of administrative rejection = 23.5%, All applications success rate = 16.2%

The differences between the White and Asian results are striking. The difference between the success rates (34.8% versus. 18.4%) is statistically significant with a p value of 0.006. The difference between the the all applications success rate (29.4% versus 13.2%) is also statistically significant with a p value of 0.0008. Finally, the difference between the probabilities of administrative rejection (15.4% versus 28.1%) is statistically significant with p = 0.007.

The differences between the White and Other category results are less pronounced but also favored White applicants.  The difference between the success rates (34.8% versus. 21.1%) is not statistically significant although it is close with a p value of 0.066. The difference between the the all applications success rate (29.4% versus 16.2%) is statistically significant with a p value of 0.004. Finally, the difference between the probabilities of administrative rejection (15.4% versus 23.5%) not statistically significant with p = 0.14 although the trend favors White applicants.

I, personally, find it hard to difficult to reconcile these data with the statements in the NIGMS Feedback Loop post. Again, this states:

We did not observe any significant differences in the gender or race/ethnicity distributions of the MIRA grantees as compared to the MIRA applicants who did not receive an award. Both groups were roughly 25% female and included ≤10% of underrepresented racial/ethnic groups.

There are statistically significant differences in the race distributions of the MIRA grantees as compared with the MIRA applications with more White compared to Asian individuals among the grantees compared to those who did not receive an award. These differences are sufficiently large that they are unlikely to be dramatically affected by the applicants with unknown or withheld race.

The statement that "both groups were roughly 25% female" is true but, from the available data, the grantee pool was 21.1% female and the pool of those not receiving an award was 27.4% female. These numbers are somewhat uncertain because of the number of applicants with unknown gender. However, there appears to be a trend disfavoring applications from females.

The data obtained through the FOIA do not allow a critical analysis of the comments about underrepresented racial/ethnic groups. However, it appears that there were no awards to applicants who identified as African American, Native American, or Multiracial, based on the missing rows in the spreadsheet that I obtained through FOIA.

Every parameter that I examined favors white or males over other groups. I find it quite discouraging that NIGMS chose to present these outcomes in a somewhat distorted and superficial manner rather than more fully presenting the data and engaging the community on trying to understand the bases for these apparent biases.

Updated:  I corrected several typographical errors.

19 responses so far

Maximizing Investigators' Research Awards for Early Stage Investigators: A High Percentage of Administratively Rejected Applications

(by datahound) Sep 29 2016

The National Institute of General Medical Sciences (NIGMS) recently launched new program the ‘Maximizing Investigators’ Research Award’ (MIRA) program including a variant for New and Early Stage Investigators. The initial results of this program have recently been released. The beginning of this announcement states (bold added):

“We received 320 applications in areas related to NIGMS’ mission, and they were reviewed by four special emphasis panels organized by the NIH Center for Scientific Review. We anticipate making 93 awards, which is more than we estimated in the funding opportunity announcement (FOA); the corresponding success rate is 29.1%.”

This language of this announcement struck me since I have heard from some early stage investigators that their applications had been administrative rejected.

Even though the fiscal year is not quite complete, I decided to request information about this program through FOIA. NIH replied promptly (within four weeks) with the requested information.

The bottom line is that is appears that 83 out of 403 applications that we submitted were administratively rejected. Of the remaining 320 applications, 93 have been or are expected to be funded.

One possible basis for administrative rejection is lack of eligibility as an early stage or new investigator. Another likely basis, given the language in the announcement noted above, is that the applications were deemed to be more appropriate for NIH Institutes or Centers than for NIGMS. This is a problem for Funding Opportunities that are specific to a single Institute or Center. Even though an application may be highly meritorious (although one does not know this since the application was not peer reviewed), it is rejected since it would normally be assigned to a different Institute or Center. With funding opportunities with broader institute participation (such as the parent R01 announcement), an application that does not fit into the areas of interest of one institute can be assigned to another, more appropriate, institute. Since this was an NIGMS-only announcement, that option was not available in this case.

Based on these data, the actual chance of having this type of MIRA application was funded was 93/403 = 23.1%.

The new Funding Opportunity Announcement for this program has been released.

Potential applicants should be very mindful of this comment in the announcement:

Research that involves a major change in scientific focus or that migrates away from the mission of NIGMS and/or into an area of major interest of one of the other NIH Institutes or Centers would warrant a discussion with NIGMS program staff.

Contact the listed program officer

Peter C. Preusch, Ph.D.

National Institute of General Medical Sciences (NIGMS)

Telephone: 301-594-0828

Email: preuschp@mail.nih.gov

to ensure that your chosen plan meets that criterion before you spend your time preparing a full application. Be very explicit about your concerns. Of course, this is good advice for any grant application.

There are more data in the FOIA response. Stay turned for more analysis.

17 responses so far

#drugmonkeyday

(by datahound) Sep 23 2016

I want to add my voice to those thanking Drugmonkey today. I started reading Drugmonkey when I was Director of NIGMS at NIH. While the blog could certainly be a bit strident, the author was generally more well informed about NIH policies, practices, and realities than almost anyone that I had met either inside or outside NIH. Over time, I started to comment on Drugmonkey posts when I thought I could add something to the discussion. This experience led me to start reading other blogs and this, in turn, let me to work with the excellent communications staff at NIGMS to launch the NIGMS Feedback Loop just in time for the ARRA funding chaos. Drugmonkey was very encouraging of this effort and we became frequent cross-commenters.

After I left NIGMS, I remained interested in blogging and was delighted when Drugmonkey and his/her colleagues at Scientopia invited me to join. After some time for consideration, Datahound was born. As this post indicates, Datahound is still alive and well although I am spending much of my blogging time at my newest blog Sciencehound. I greatly appreciate the personal, if mostly indirect, mentoring I have received from Drugmonkey as well as the service Drugmonkey provides to the scientific community in general (especially early career folks). Thanks!

5 responses so far

Datahound's New Littermate...Sciencehound

(by datahound) Aug 04 2016

It has been a bit over 2 years since Datahound was born. I have greatly enjoyed being brought into the Scientopia team and sharing data that I have gathered and analyses that I have done with my readers and plan to continue to do this going forward as my time permits. Today, I want to share some exciting news. Datahound has a new littermate...Sciencehound. I have started this new blog to share my thoughts about the communities of science similar to those I have shared here but also extending more deeply into scientific publishing as part of my new role as Editor-in-Chief at Science and the Science family of journals. Come on over and join the discussion!

No responses yet

R01 Size Growth and the Modular Cap-2016 Edition-Part 1

(by datahound) May 27 2016

Two years ago, I posted an analysis of the distributions of the sizes of R01 awards from NIH and the relationship of this distribution to the cap of $250,000 direct costs on applications submitted with modular budgets.

The recent activity with regard to compensation for post-doctoral fellows got me thinking about this topic again. Here, I update the analysis with data from fiscal year 2015. The distribution of R01 annual total cost sizes (for a total of 22588 new (Type 1), competing renewal (Type 2) and non-competing renewal (Type 5) awards) is shown below:

Total Cost Dist Figure-2015_1

Unlike the corresponding plot from fiscal year 2003, this distribution is decidedly not Gaussian. It is bimodal with peaks near $330K and $380K and has a broad tail extending from approximately $420K to $800K. The bimodal nature was subtly present in the distribution for fiscal year 2013 but is now much more pronounced. The tail had grown substantially from fiscal year 2003 to fiscal 2013 and appears to be approximately the same size in the fiscal year 2015.

This distribution can be analyzed further since RePORTER now includes information about direct costs in addition to total costs. The distribution of annual direct costs is shown below:

Total-Direct Cost DIst Fig

Examination of the direct cost data reveals several important points. First, the bimodal distribution is present just at and below the modular cap of $250K. Thus, it is not the modular cap that is leading directly to this split distribution, but rather some other factor as will be discussed below. Second, the median for the overall distribution is exactly at the modular cap level. Thus, it appears that half of the funded R01s were submitted with modular budgets and the other half were not.

What is responsible for the bimodal distribution? One possibility is that is has to do with the number of years for which the grants have been in existence. Examining the distributions of the years of support for the first peak (from $190K up to but not including $230K) and the second peak (from $230K to $250K) reveals no difference with a correlation coefficient of 0.9986.

A second possibility is that the bimodal nature is due to different policies from different institutes and centers with regard to cutting awarded grant budgets. The number of grants in the second peak ((from $230K to $250K) versus the number of grants in the first peak ((from $190K to $230K) for each institute and center is shown below:

Peak1_2 figure

This plot clearly shows that differences in institute behavior is responsible. Some institutes such as NHLBI (HL), NIAID(AI), NIMH (MH) and NIDCR (DE) have many more grants in the peak with larger grant sizes, consistent with the fact that these institutes have tended not to cut grant budgets when the grants are first awarded. Other institutes such as NIGMS (GM), NCI (CA), NINDS (NS) and NIDDK (DK) have more grants in the peak with smaller grant sizes, consistent with the fact that these institutes have tended to make such budget cuts.

It is remarkable that this bimodal behavior has emerged from just a hint in the data from fiscal year 2013 to absolute clarity two years later. At NIH, each institute is given considerable autonomy to make and implement such policies.The clean separation of these two groups of institutes strongly suggests that the cap on modular budgets is distorting grant sizes because many applications are submitted at the modular cap level and then cut upon award by the funding institute or center. Furthermore, these data reveal some consequences of this where, for example, similar grants awarded by NINDS and NIMH could be funded at substantially different levels.

3 responses so far

The NIH Early Career Reviewer Program-Some Key Parameters

(by datahound) Feb 05 2016

Many have stated that serving on a study section can be an important step developing in an academic career. I did not serve until I was relatively well established, but I do vividly recall that the opportunity helped me understand the process better and improved my ability to craft better proposals. Moreover, it increased my faith that the peer review system was thoughtful and fair although certainly not perfect. In 2011, NIH announced a new program to allow early career scientists to serve on study sections. This program has two primary objectives. First, it allows early career investigators to observe and understand the review process directly with the goal of helping them enhance their abilities to write more competitive proposals. Second, it helps NIH review staff identify and screen potential reviewers for additional service.

To be eligible for the Early Career Reviewer Program, a individual must not have had substantial NIH review experience and have at least 2 years experience as an independent investigator with 2 recent publications as a senior author.

Following a recent discussion on Twitter about how many individuals who had applied to the program had actually served on a study section, I submitted a FOIA request for data regarding the program. I recently received the response. While the data are not comprehensive, they do allow an assessment of some of the key parameters.

(1)  The acceptance rate into the program appears to be around 64%. Specifically, from the period from June 2014 through December 2015, NIH received 1863 applications for the program and accepted 1189 individuals. The total number of applicants to date appears to be 4,534.

(2) The number of individuals who have actually served on a study section in this capacity is 1706 from the beginning of fiscal year 2012 when the program began through the first round of fiscal year 2016.

Graphically, the progression through the program is as follows:

ECReviewer plot

Three additional parameters are:

(3) The percentage of women accepted into the program is approximately 45% and a similar percentage of women have actually served.

(4) The pool of scientists accepted into the program is approximately 6% African American, 7% Hispanic, and 27% Asian.

(5) Approximately 13% of the individuals accepted into the program are from institutions eligible to apply for R15 (AREA) grants.

I do not know if NIH is planning an evaluation of this program. However, such an evaluation would seem to be straightforward and would help codify the value of study section experience to scientists developing their career and could also address the question of how well early career scientists perform compared to more established scientists in the review process. What are your thoughts?

 

11 responses so far

NIH High Risk Research Programs: Racial Composition of Applicant and Awardee Pools

(by datahound) Jan 17 2016

To continue my series of posts on the NIH HIgh Risk Research programs, I submitted a FOIA request to NIH for data regarding the race and ethnicity of the applicant pools for the four High Risk Research programs and recently received the results.

These data represent voluntarily self-identified race and ethnicity data that are collected when initiating an NIH Commons account (and used to be collected as part of an application). The categories listed are African American, American Indian, Asian, Multiple Races, Native Hawaiian, Unknown, White, and Withheld. While the data are presented by year (as I had requested), data are redacted for cells that represent fewer than 11 individuals. Since this limitation applies to many cells, I will focus on the data aggregated over all years of each program. It is important to note that these are not unique applicants, that is, applicants who apply to a given program multiple years will be counted multiple times.

The results for the African Americans, Asians, White, Unknown, and Withheld categories are shown below. The results for American Indians, Multiple Races, and Native Hawaiians are not included in this graph because of the redacted data.

Application pools

These data show that the percentage of White applicants ranged from 53% to 68% with the highest percentages for the Pioneer and Transformative R01 programs (68% and 63%) and lower percentages for the New Innovator (53%) and Early Independence (56%). This correlates with the likely career stage of the applicants with the Pioneer and Transformative R01 programs attracting established, more senior, investigators. The percentage of African American applicants ranged from 1.4% to 2.7% with the opposite trend (higher percentages for the New Innovator and Early Independence programs). The percentage of Asian applicants ranged from 16% for the Early Independence program to 30% for the New Innovator program with those for the Pioneer (18%) and Transformative R01 (20%) at intermediate values.

How do the applicant pools compared with the awardee pools? Because of the relatively low number of awardees for these programs, data about the race and ethnicities of these individuals were inferred by examining each awardee individually. It is important to note the difference in methodology between self-reported race and ethnicity for the applicant pool and inferred race (based on available biographical information, appearance, and name) for the awardee pool.

With this important disclaimer, I estimated the numbers of African American, Asian (primarily Chinese, Indian, and Japanese), and Other awardees in each program. These were compared with the corresponding numbers from the applicant pools. The ratios of Awardees to Applicants for these three racial groups for each program are shown below. The numbers of applicants in each group is also shown since these bear on the interpretation of the results.

Success_Rates

For the Pioneer program, the ratio of awardees to applicants is 8/64 = .125. This ratio is higher than the ratio for Other (non-African American, non-Asian) at 107/3053 = 0.035. This difference is statistically significant with a p value of 0.002. Similarly, the ratio for Asians of 40/693 = 0.058 is significantly larger than that for Other with a p value of 0.009. The difference between the ratios for African Americans and Asians is not statistically significant (p = 0.054).

These results suggest that the success rates for African Americans and Asians are higher than that for Other applicants. Interpretation of these observations must be done with considerable caution. First, the number of African American applicants and awardees is quite small, an average of less than 1 awardee per year. While the result is statistically significant, it is not very robust to small changes in the number of awardees. Second, there is considerable selection bias in this program, based on my direct experience trying to encourage individuals to apply. Some are not aware of the program (or the timeline) while others feel that they are so unlikely to succeed that they are reluctant to apply even when encouraged. This self-selection could apply somewhat differently to different racial groups although I have no data that bear of this.

For the New Innovator program, the observed ratio of awardees to applicants is lowest for African Americans, slightly higher for Asians, and highest for Other, but these differences are not statistically significant. The numbers of applicants in all three groups are relatively large so that these findings are more robust than those for the Pioneer program.

For the Early Independence program, there appear to have been no African American awardees. However, there have been only 10 self-identified African American applicants (this value was not redacted in the materials I received) and the difference between the ratio of 0/10 is not statistically significantly different from 59/381 for Other (p value 0.37). The ratio for Asians is slightly higher than that for Other, but this difference is also not statistically significant.

The analysis for the Transformative R01 program is slightly more complicated because this program allows multiple principal investigators (PIs). I have included all PIs in my analysis of the awardees and believe this is also true for the applicant data that I received from NIH (although I am working to confirm this). With those caveats (and the disclaimer above about methodology), the ratio of awardees to applicants is 2/75 for African Americans. This ratio is slightly lower than the ratio of 154/4099 for Other, but this difference is not statistically significant (p = 1.00). The ratio for Asian applicants is also not statistically different from the other ratios.

As a final note, there is some overlap between the awardees in the Pioneer, New Innovator, and Transformative R01 programs with awardees from one program going on to receive additional awards in the same or other programs. Indeed, both African American Transformative R01 recipients had previously received Pioneer awards. This overlap will be the subject of a future post.

3 responses so far

Gender Balance in NIH High Risk Research Programs-Overall Award Pool versus Applicant Pool

(by datahound) Dec 10 2015

I recently posted analysis regarding the gender distribution between the awardees and the applicant pools for the four NIH High Risk Research (Pioneer, New Innovator, Early Independence, and Transformative R01) programs for 2015. Data regarding the awardees can be gleaned from NIH RePORTER or the NIH Common Fund website. However, data regarding the applicants are not available except through Freedom on Information Act (FOIA) requests. This is how I obtained the data for the 2015 applicants. I subsequently submitted a request for data regarding the applicants pools for all years since each award program was created. While I intended to get data for each year separately, I was not sufficiently clear in my request and I instead received aggregate data over all years. I have clarified my request and am awaiting a response. In the meantime, I want to share the analysis of the aggregate data.

Here are the data:

Program                     Applicants (Male)   Applicants (Female)       Applicants

.                                                                                                         (Gender not given)

Pioneer                              2051 (75.4%)           666 (24.6%)                    53

New Innovator               3498 (67.2%)         1706 (32.8%)                   120

Early Independence        233 (59.9%)            156 (40.1%)                    62

Transformative R01      3245 (78.4%)            894 (21.6%)                  160

 

A glance at these data reveals a couple of points. First, the gender balance of the applicant pool follows a pattern that could be anticipated based on the career stage mix with the closest balance for the Early Independence ("Skip the postdoc") program, followed by the New Innovator Award program, the Pioneer program, and, lastly, the Transformative R01 program. Second, while the number of investigators with genders withheld or unknown is less that 4% for three of the programs, it is 13.7% for the Early Independence program. This introduces some uncertainly in the analysis as will be discussed later.

How does these results compare with the Awardee pools?

Program                        Awardees (Male)        Awardees (Female)

Pioneer                              110 (71.0%)                   45 (29.0%)

New Innovator               263 (66.0%)                 135 (34.0%)

Early Independence         54 (75.0%)                   18 (25.0%)

Transformative R01       164 (83.2%)                   33 (16.8%)

 

Let's compare the applicant pool with the awardee pool program by program. For the Pioneer program, women correspond to 24.6% of the applicant pool and 29.0% of the awardee pool. This difference is not statistically significant with a p value 0f 0.18.

For the New Innovator program, women correspond to 32.8% of the applicant pool and 34.0% of the awardee pool. This small difference is not statistically significant with a p value of 0.62.

For the Early Independence program, women make up 40.1% of the applicant pool but only 25% of the awardee pool. This difference is statistically significant with a p value of 0.0049. Recall that 62 of the Early Independence applicants had genders that were unknown or withheld. If we assume that these applicants were split with the same proportions as the remainder of the pool (60% male, 40% female), the p value is decreased slightly to 0.0040. If we assume that the applicants of unknown gender were 50% male, 50% female, the p value is increased t0 0.0090. Only if we assume that all or nearly all of the applicants of unknown gender were male (and there is not reason to think that this is true), does the p value go above 0.05 (p value = 0.078 if all are assumed male). Thus, it appears very likely that there is a statistically significant decrease in the proportion of women in the Early Independence awardee pool compared with the applicant pool. This observation supports my proposal that NIH should investigate this program carefully to try to understand the source(s) of this disparity.

For the Transformative R01, women make up 21.6% of the applicant pool and 16.8% of the awardee pool. This difference is not statistically significant with a p value of 0.092. However, it is noteworthy that both the applicant pool and the awardee pool are strongly dominated by male principal investigators. This may reflect the relative seniority of applicants to this program or other factors about the program structure or management. This will require more analysis.

These results are summarized in the figure below:

High Risk Programs-Gender

I await the data broken down by year to see if there are meaningful trends over time. In the meanwhile, these data do provide the scientific community and the NIH some food for thought.

8 responses so far

Nelson Diversity Surveys: A Rich Data Source regarding Women and Minorities in Science

(by datahound) Dec 03 2015

The diversity (or relative lack thereof) across university campuses is a central issue in academia. Many institutions have ongoing diversity programs include several high-profile institutions (e.g. Yale, NYU, Brown) that have recently announced new initiatives. The diversity of the faculty represents a crucial subset of this issue and the diversity of the science and engineering faculty is an important component of this.

The development and implementation of programs intended to increase diversity depends, in part, on the availability of data to provide baselines and to monitor progress. The university (as opposed to medical center) based community is blessed with the Nelson Diversity Surveys, robust data sets regarding the representation of women and minorities in university science and engineering departments, generated through the efforts of Professor Donna Nelson and her colleagues. These surveys include essentially complete information regarding the number of faculty members in the top (in terms of federal funding) 50-100 departments in  Chemistry, Biological Sciences, Astronomy/Astrophysics, Physics, Mathematics/Statistics, Computer Science, Psychology, Political Science, Sociology, Economics, Chemical Engineering, Electrical Engineering, Civil Engineering, Mechanical Engineering, and Earth Sciences, broken out by rank, gender (Male, Female) and race/ethnicity (White (non-Hispanic), Black, Hispanic, Asian, and Native American). Importantly, these data are not samples, but complete snapshots taken in 2002, 2007, and, more recently 2012) obtained by polling and relentlessly following up with each department. The reports including data for 2002 and 2007 are freely available and the 2012 data should soon be available once the data validation and initial analysis are complete. I first became aware of these data when I was at NIGMS and we subsequently had Professor Nelson speak at one of our Advisory County meetings.

To give an example of the richness of these data, one Table (for the top 50 chemistry departments in 2007) is shown below. Note that the numbers presented as decimals represent women faculty members.Chemistry-Nelson-07Report-RotatedSome overall statistics for this subset of the Nelson Diversity Survey can be derived. Overall, these chemistry faculty were 13.7% women, 1.6% Black, 2.2% Hispanic, 10.0% Asian, and 0.2% Native American. These data can be compared with other population statistics. For example, the percentages of Ph.D.s awarded in Chemistry from 1995-2006 (also provided in the report) were 32.4% for women, 3.5% for Blacks, 3.4% for Hispanics, 12.8% for Asians, and 0.6% for Native Americans. Thus, the data reveal (not at all surprisingly) that the representations of all of these groups within the faculty is lower than they are in the recent Ph.D. population.

The availability of data broken down by department makes it possible to examine these averages in much more detail. The departments ranged in size from 18 to 56 faculty with a median of 33. Below is a histogram of the number of departments versus the percentage of women in each department. For comparison, a Poisson distribution with a mean of lambda = 13 (the distribution that would be expected if recruitment of women faculty into a given department were random with the same overall rate).

Women-Hist-Poisson

The fit to the Poisson distribution is approximate, but with substantial probability density moved out of the center toward departments with lower or higher percentages of women. Being cautious not to over-interpret this small data set (although it is certain possible to examine other fields to see if similar trends are observed), this supports the hypothesis that departments that already have a higher percentage of women tend to recruit more women (keeping in mind the low overall percentages for chemistry departments in general).

Below are histograms depicting the number of departments with a given percentage of faculty in each minority racial/ethnic group.

Chemistry_Histograms

62% or more of the departments did not have a single Black faculty member. Similarly, 50% of the departments did not have a single Hispanic faculty member. The distribution for the percentage of Asian faculty members resembles that for women with an approximately Poisson distribution with some probably density moved out of the center. Only 3 departments identified have even one Native American faculty member. 13 of the 50 departments did not have a single Black, Hispanic, or Native American faculty member.

This quick pass through a small subset of the Nelson Diversity Surveys reveals some of the potential for exploring details and developing and testing hypotheses. Expanding this across different fields and, most interestingly, across time with the release of the 2012 data, should provide insights to help focus discussions of diversity and implementation strategies for improving diversity for the benefit of all.

 

Update:  Dr. Nelson informed me that the 2012 results will be released at the American Chemical Society meeting in March.

2 responses so far

NIH "High Risk" Programs-Part 3-New Innovator Award Program

(by datahound) Oct 26 2015

I have previously discussed the history of the NIH Director's Pioneer Award program. The daughter program of the Pioneer program is the NIH Director's New Innovator Award program. This program was created in 2007 when Congress was working on an appropriations package in February for the fiscal year that started the previous October. I was involved in the conception of this program and I recall quite clearly how I was sitting in a meeting when someone from the NIH Director Elias Zerhouni's office came in and asked if I could spare a few minutes to speak with Dr. Zerhouni and the NIH Budget Director. Needless to say, I said that I could. In the course of the budget negotiations, Congressional members and staff had heard Dr. Zerhouni's strong concern about the plight of young investigators and had been pleased with the initial progress with the Pioneer program. They wanted to know if NIH could establish a "junior Pioneer" program if they provided some additional funding. I was asked what I thought of the idea and if we could set up such a program. Again, needless to say, I indicated that I thought we could do that. The bill was passed with $40M for this new program in the budget for the NIH Common Fund.

This was mid-February and this program has to be set up from scratch and operationalized to get these funds out the door by the end of the fiscal year (September 30, 2007). After the meeting, I walked back to NIGMS and found Judith Greenberg and others involved with the Pioneer program (which was now being administered through NIGMS) and told them that I had so much faith in them that I had committed us to build this new program on this very short timeline. I am not sure they were entirely thrilled with me but, as always, they rose to the occasion and got right to work on the many tasks that needed to be tackled.

The first key question was how to target the program to young investigators since basing eligibility on age was clearly not legal. We came up with the idea of using time since an individual had received their "terminal degree" (typically PhD or MD). But, what was the appropriate time period? After some discussion and a little analysis, we settled on 10 years (with some exceptions for clinical training or time off for family responsibilities). This decision became the basis for the subsequent NIH definition of an "early stage investigator (ESI)".

The second key question was how large the awards should be. The NIH Budget Director John Bartrum had very cleverly required that this awards be made with a new mechanism, the DP2. This mechanism is quite unusual in that it allows NIH to commit funds for multiple years from the same fiscal year. This was done to avoid adding to the commitment base with these awards. Normally, when an institute funds a multi-year award in one fiscal year, this commits the institute to funding out years of the award in subsequent fiscal years. Since NIH does not know the appropriation level for the next fiscal years, this can be a real challenge. Making lots of commitments and then receiving a poor appropriation the next year can limit the number of new and competing grants substantially, leading to low success rates and other issues. With the new DP2 mechanism, NIH could fund each entire award out of one fiscal year without taking on any new commitments. The decision was made that the awards be $1.5 M direct costs over 5 years (but paid to the institution in the first year). Thus, with $40M and an estimated average grant size of $1.5M direct plus ~$1.M indirect) = $2.5M, we expected to be able to make about 14-16 awards.

The next challenge was writing the Request for Applications and getting the word out about this program so that eligible individual would have some time to get their applications conceived and submitted. It is a testament to Judith Greenberg and her coworkers that the RFA was published on March 9, 2007, approximately 3 weeks after the bill was passed. Trust me, the NIH bureaucracy does not normally work that fast. We worked with our communication staff to publicize the funding opportunity through every reasonable channel we could think of since our fear was that eligible folks would not find out about the program in time to submit applications. Applications were due on May 22 with this short period dictated by the need to get the applications processed, reviewed, and funding decisions made by September 30. Our publicity approach was successful when we realized that we had 2153 applications in by the deadline. This, of course, created a new challenge with funds for 14 awards and 2153 application leading to a projected success rate of a whopping 0.6%.

We had taken the lessons that we had learned through the Pioneer program and made sure that all stages of the process emphasized that innovative research could be expected from scientists of all genders and from diverse backgrounds. The review process (again, set up on a short timescale) involved a 2 phase electronic review. No interviews were included (in distinction to the Pioneer award). We spent considerable time orienting each group of reviewers so that they understood the vision for the program in all aspects. When the reviews were completed, we has an outstanding set of highly ranked applications. I set to work soliciting funds from institutes to increase the number of awards. This was challenging since these awards cost about $2.5M a pop but the lack of out-commitments was a selling point. By September 30, we were able to make a total of 30 awards to some outstanding scientists including 12 women and 18 men in a range of fields. I had the privilege of personally calling these individuals to tell them that they would likely be receiving these awards (and getting some additional information to make sure that they were still eligible).

Based on the successful launch, funds were provided for the New Innovator program in subsequent years. After the program had been running for three years, we initiated an outside process evaluation, parallel to the one for the Pioneer program. This report contains considerable information about the program, including comparisons of the applicant, finalist, and awardee pools.

New Innovator Gender

There were no statistically significant differences between the compositions of the applicant and awardees pools with regard to either gender or race/ethnicity.

The New Innovator program continues to the present with 497 applicants and 41 awards in fiscal year 2015. The goal of enabling young scientists to get off to a running start in hugely important and others feel the same way. Bruce Alberts, when he was Editor of Science magazine (and a reviewer for the New Innovator program) called for increasing the number of awards to 500 per year. One of the most touching (but also distressing) experiences that I had with this program was receiving several emails from unsuccessful applicants telling me how much fun it had been to write an application about what they actually want to achieve as opposed to what they thought they could get funded to do. This seems to me to be a significant indictment of the current state of affairs. In addition, of course, it is essential that support is available to sustain these careers once they are launched.

6 responses so far

Older posts »