Reforming (funding) peer review

July 16, 2008

NIH has announced an initiative to enhance its peer review practices. In so doing, the world’s largest science funding agency acknowledges the validity of two criticisms often levied by grantees and science policy-types:

  • that the NIH discriminates against young researchers by putting a premium on preliminary results when evaluating proposals. This practice has created a situation in which funding is better viewed as an intermediate research output – a prize for work well done, in the words of Ed Lazear – rather than an input into the production of future results.
  • that peer review as practiced by the CSR (Center for Scientific Review) results in the funding of incremental, “normal science”, at the expense of truly creative proposals.

Both criticisms have some merit, and we’ll discuss them (separately) at some later date. For now, I’d like to make a very simple point. To paraphrase Churchill, peer review is a terrible system to allocate funding across projects…except that all the others are worse still. The alternatives, such as an institute model (as practiced in France, or in many of the US National Labs) can easily produce fiefdoms insulated from competition and increasingly disconnected from the outside world. There are also prizes, but they seem much more appropriate to reward innovation towards goals well-specified in advance. In short, there are many reasons to suspect that the competitive, peer-reviewed grant is an important meta-institution whose adoption greatly increased the world’s epistemic base.

So I am hoping that the NIH task force will remember a few things as it develops its reform proposals further. First, do no harm! Remember that those who have the most gripes against the current system might have agendas of their own. Second, experiment! Although scientists often have strong priors regarding what is likely to work, subjecting reform proposals to careful, randomized evaluations is the way to go…


Why Innovation Today

July 16, 2008

First of all, thanks for getting the ball rolling on this, Pierre. It is particularly timely to be thinking about innovation now with the increased economic strength in Asia making innovation particularly critical for us, the aging of the workforce – I think that most people think of innovation as done by relatively young people; and energy prices increasing.

On the day when we found out that consumer prices have risen more in the past year than at any point since 1991, in large part because of the price of energy, one idea that I am intrigued by is that increased investments in energy technology now could lead to lower prices now even though they won’t pay for many years because it would make oil producers want to pump oil now rather than later.

Inaugural Post

June 29, 2008

Welcome! We are a group of social scientists interested in the economics of science and technical change, broadly construed. Come back often to read our musings on everything patent, the institutions of science, demand-side increasing returns, and arcane data and econometric issues. Besides commenting on science and technology policy, we will use this blog to showcase our research and that of our “invisible college.” You can also count on us to engage in wild speculations — precisely what we would not be able to do in a typical scholarly article.