EA - Doing EA Better: grant-makers should consider grant app peer review along the public-sector model by ben.smith

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Doing EA Better: grant-makers should consider grant app peer review along the public-sector model, published by ben.smith on January 24, 2023 on The Effective Altruism Forum.Epistemic status: speculativeThis is a response to recent posts including Doing EA Better and The EA community does not own its donors' money. In order to make better funding decisions, some EAs have called for democratizing EA's funding systems. This can be problematic, because others have raised questions such as (paraphrasing heavily) such as: "how do we decide who gets a vote?" and "would funders still give if they were forced to follow community preferences"? The same EAs have argued that EA decision-making is “highly centralised, opaque, and unaccountable”, and said that to improve our impact on the world, the effective altruism movement should be more decentralized and there should be greater transparency amongst EA institutions.To meet both of the families of concerns expressed over the last week, I propose a grant-assessment system that improves transparency, decentralizes decision-making, and could better inform grant allocation by drawing information from a wider section of the community whilst maintaining funders' prerogatives to select the areas they wish to donate to. The proposal is to adopt a peer-review process used by the grant-making system run by public bodies in the United States, such as the National Institutes of Health and the National Science Foundation.In this model, the funder’s program manager makes decisions about grant awards based on reviews and numerical scores allocated by peer reviewers coordinating in expert panels to evaluate grant applications. This would be a positive-sum change that benefits both funders and the community: the community has more input into the grant-making process, and funders benefit from expertise in the community to better achieve their objectives.In the rest of this post, I will describe the National Institutes of Health grant evaluation process, describe why I think now is the right time for the effective altruism movement to consider peer review as part of a more mature grant evaluation process, give some notes on implementation in EA specifically, and describe how this approach can both maintain funders’ prerogative to spend their own money as they wish, while giving the community a greater level of decision-making.The grant peer review process at the NIH and NSFNational Institutes of HealthThe National Institutes of Health (NIH) uses a peer review process to evaluate grant applications. This process involves the formation of ‘study sections’, which are groups of experts in the relevant field who review and evaluate grant applications.When an application is received, it is assigned to a study section based on its scientific area of focus. Each study section is composed of scientists, physicians, and other experts who have experience in the field related to the research proposed in the application. These would be drawn from the scientific community at large. Study section members are typically compensated for participation, but participation isn’t a full time job–it’s generally a small additional duty researchers can choose to take on, as part of their broader set of research activities.The study section members more-or-less independently review the applications and provide written critiques that are used to evaluate the strengths and weaknesses of each application. The study section then meets to discuss the applications, and each member provides a priority score and written summary of the application. These scores and summaries are used to determine which applications will be funded.In summary, the NIH uses study sections composed of experts in the relevant field to review and evaluate grant applications through a peer rev...