Archive for: March, 2015

FOIA-The Freedom of Information Act

Mar 31 2015 Published by under Uncategorized

As a datahound, I often would like access to data. Ideally, relevant data are available through published reports, websites, or publicly available databases such as NIH RePORTER. However, in many cases, such data are not available. Fortunately, Congress passed and President Johnson signed the Freedom of Information Act (FOIA) in 1966. FOIA allows citizens to request record from any executive branch agency. The government much supply these records subject to a series of exceptions including national security,  personnel records, many pre-decisional memoranda, and several other categories.

A FOIA request must be relatively specific regarding the information requested. Importantly, FOIA applies only to records that already exist. A citizen cannot request (through FOIA) that an agency conduct an analysis that is has not already been done, but can request materials related if s/he knows that they exist.

Each agency has its own FOIA process. For example, information about the NIH FOIA process including how to file a request can be found here. I first used the FOIA process in an attempt to obtain information about the decision by the NIH Director and the Secretary of Health and Human Services to dissolve the National Center for Research Resources. In this case, the process was quite slow and frustrating. More recently, I use FOIA to obtain information about the age distribution of NIH New and Early Stage Investigators. In this case, the process was more rapid (although still took a couple of months to get the data) and I received almost all of what I requested.

My most recent FOIA request relates to the discussions of a potential emeritus award from NIH. Conversations with colleagues at the Experimental Biology meeting indicated that some NIH staff members had indicated that the responses to the NIH Request for Information (RFI) had been more positive that those to the Rock Talk post on the subject (which were almost uniformly negative). I have requested the responses to this RFI. Time will tell what I receive and what these records reveal about the respondent's thoughts are about this possible award.

Update:  The initial response is below:

FOIA-RFI-4-15

17 responses so far

Selected Data and Sources Relevant to Research Enterprise Sustainability

Mar 28 2015 Published by under Uncategorized

I participated in the follow-up meeting to the Alberts et al. paper that was summarized in a recent PNAS paper. This summary noted that "...most were surprised to learn that the percentage of NIH grant-holders with independent R01 funding who are under the age of 36 has fallen sixfold (from 18% to about 3%) over the past three decades." This statement is probably accurate, but I was disappointed that many participants were not familiar with many important facts and trends that have affected the biomedical enterprise over the past two decades.

What information is important for individuals to know in order to participate in discussions about potential corrections to the present system. In addition to the demographic data noted above, below are some slides that I have used in presentations on the topic of the sustainability of the biomedical research enterprise (some of which are derived from posts here or from my columns at ASBMB Today.

Sustainability-1

Slide01

Slide02

Slide03

Slide04

Slide05

Slide06

Slide07

Slide08

Slide09

Slide10

Slide11

 

Which of these are most important? What are other important data sets or data sources that should be included in such presentations?

18 responses so far

More Age Data from NIH: Surprising Award Rate Data

Mar 27 2015 Published by under Uncategorized

In the context of recent discussions of NIH age group data, @dgermain21 pointed to some interesting data in a recent NIH report on physician scientists regarding NIH R01 Award Rates as a function of age group (as well as degree, race/ethnicity, and gender. These data are quite surprising as shown below:

Award Rate graph

These data are for all individuals in the analysis. I have omitted the curves for individuals 30 or less and 71+ since these data are relatively noisy, presumably to relatively small numbers of individuals in these groups.

The term Award Rate is defined by NIH as "the number of awards made in a fiscal year divided by the absolute number of applications where we don’t combine resubmissions (A1s) that come in during the same fiscal year." Thus, Award Rate is lower that Success Rate since the denominator is higher.

Of course, the surprising observation is that these rates are highest for the 31-40 age and declines monotonically so that it is lowest for the 61-70 age group. This is certainly counter to what I would have expected where I would have anticipated the opposite trend or perhaps a peak for the 51-60 age group. This observation begs an explanation.

Digging into the report, the Quantitative Analysis Methodology section indicates that

"The NIH awards and time period selected for inclusion in the system from IMPACII (the large internal NIH database) were:

  • Research Project Grants for the following 25 activity codes between 1993 and 2012, Type 1 applications,..."

The term "Type 1" applications refers to new (as opposed to competing renewal) applications. This suggests that the above data may be only for these new applications. Competing renewal applications (Type 2) applications come predominantly from more senior investigators and have substantially higher success rates than new applications. Thus, the restriction to Type 1 applications would be expected improve the importance of younger relative to older investigators. This may be an important contributor to these data, although I still find it surprising that the reported trends still apply to new R01 applications.

Interested readers should look at the report and help try to understand how to interpret these data.

UPDATE

I contacted the individuals responsible for the data in this report. The missing data (zeros for PIs between 61 and 70) have been filled in, the the NIH believes that the other data are correct as posted. Thus, it appears that award rate for new (Type 1) R01s appears to decrease monotonically with increasing PI age and this was true for every year from 1999 to 2012.

7 responses so far

Rock Talk Age Data: Effective Indirect Cost Rates 1998-2014

Mar 26 2015 Published by under Uncategorized

A recent post on Rock Talk presented data on the amount of funding as a function of PI age group. These data were not presented in a terribly informative way but a file was available for downloading and Michael Hendricks normalized the data by the number of PIs in age group to reveal more interesting trends, discussed at Drugmonkey.

The downloadable data includes a breakdown of Direct and Total Costs. I have been looking for such data over a longer period than the last couple of year and thought I would take a look. Below is a plot of the Effective Indirect Cost Rate ((Total Costs-Direct Costs)/Direct Costs) for the overall data set.

Overall indirect graph

The Effective Rate drops from 44.2% to a low of 37.2% in 2012 before rising slightly over the past two years. These values are all somewhat lower than I anticipated based on my previous analysis on R01s.

To try to gain some insight, I looked at these data as a function of PI age group.

Indirect cost graph

The differences between the age groups quite substantial and surprising. For the lowest three PI age groups, the Effective Rate is relatively constant around 47%, consistent with my previous R01 indirect cost analysis. For the older PI age groups, the Effective Rate falls steadily from 1998 to 2012, reaching rates as low as 27.5% for PIs 61-65 in 2012.

I certainly do not understand what underlies these trends, but differences in mechanisms could certainly be involved. It may be that mechanisms as as U01s for larger efforts could be important. As always, it would be best to see data broken down by mechanism to facilitate accurate interpretation.

Any other thoughts on these data are most welcome.

20 responses so far

NIH Institute and Center Strategic Plans

Mar 17 2015 Published by under Uncategorized

The National Institute of General Medical Sciences (NIGMS) just released its new strategic plan. I was involved in writing the first NIGMS Strategic Plan released in 2008. I must admit that I was skeptical of the value to writing a strategic plan when we started down the path toward this plan. However, I found both the process and the final product useful. It allowed the institute to formalize our commitment to research and training activities, the balance of the use of particular mechanisms, and so on. Even though many aspects of the plan seemed obvious in terms of the culture of NIGMS, I found myself looking back to the plan and referring others to important sections over the next years.

The new NIGMS plan takes a similar approach. One section of interest is:

Objective 1-2: Promote the ability of investigators to pursue new research directions, novel scientific insights and innovative ideas.

This objective includes the following implementation strategies:

1. Align funding decisions with the need to create a broad and diverse research portfolio that maximizes the scientific return on taxpayers’ investments.

2. Support investigators working in underexplored areas of science that are relevant to the NIGMS mission.

3. Conduct regular analyses of NIGMS’ scientific investments to assess their efficacy, distribution and impact, and use this information to help guide programmatic and funding decisions.

4. Pilot and assess alternative mechanisms of funding that emphasize individual investigators, rather than individual projects, to better meet NIGMS goals and objectives.

5. Increase the Institute’s ability to conduct in-depth portfolio analyses and evaluation activities.

Importantly, this includes commitments to analysis and analysis capabilities (strategies 3, 5) as well as a strategy related to the new Maximizing Investigators' Research Award (MIRA) mechanism.

While I am still going though the plan, plans for other NIH Institutes and Centers are available (although some links on this site are broken).

Do you find these plans of any use? How could they be made more useful?

15 responses so far

The NIH Office of Extramural Research (OER)

Mar 02 2015 Published by under Uncategorized

In many discussions including a recent one at Drugmonkey, issues around the role of the NIH Deputy Director for Extramural Research (DDER) (a position currently held by Sally Rockey) have arisen. DDER is a big job with many responsibilities. Below is the Organizational Chart of OER.

OER_main

 

As you can see, OER is responsible for including extramural policies and programs, data analysis and sharing (e.g. NIH RePORTER), laboratory animal welfare, and administrative operations.

OER has a limited role in communications, specifically related to extramural research. This role expanded with the founding of Rock Talk blog. As Sally Rockey has credited the NIGMS Feedback Loop blog which I started when I was at NIGMS as a model for this blog, I take some pride and responsibility for the existence of this blog. With that said, I share many of the concerns that have been raised about NIH's reactions to comments of this blog.

8 responses so far