An experimental test of the effects of redacting grant applicant identifiers on peer review outcomes
- Retired, United States
- NIH/Center for Scientific Review, United States
- National Institutes of Health, United States
- Evaluact, Inc, United States
Abstract
Background: Blinding reviewers to applicant identity has been proposed to reduce bias in peer review.
Methods: This experimental test used 1200 NIH grant applications, 400 from Black investigators, 400 matched applications from White investigators, and 400 randomly selected applications from White investigators. Applications were reviewed by mail in standard and redacted formats.
Results: Redaction reduced, but did not eliminate, reviewers' ability to correctly guess features of identity. The primary, pre-registered analysis hypothesized a differential effect of redaction according to investigator race in the matched applications. A set of secondary analyses (not pre-registered) used the randomly selected applications from White scientists and tested the same interaction. Both analyses revealed similar effects: Standard format applications from White investigators scored better than those from Black investigators. Redaction cut the size of the difference by about half (e.g. from a Cohen's d of 0.20 to 0.10 in matched applications); redaction caused applications from White scientists to score worse but had no effect on scores for Black applications.
Conclusions: Grant-writing considerations and halo effects are discussed as competing explanations for this pattern. The findings support further evaluation of peer review models that diminish the influence of applicant identity.
Funding: Funding was provided by the NIH.
Data availability
All data analyzed for the findings presented in this manuscript are included in the supporting files
Article and author information
Author details
Mark D Lindner PhD
Funding
National Institutes of Health (none)
- Richard Nakamura
Employees of the NIH were involved in study design, in data analysis, data interpretation and manuscript writing. Data were collected, and major data analysis completed, by a contract research organization.
Ethics
Human subjects: All participants gave informed consent to participate in this study in accordance with a protocol that was approved on March 27, 2017 by the Social Solutions, Inc. IRB, (FWA 00008632), protocol #47.
Copyright
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Metrics
-
- 4,827
- views
-
- 584
- downloads
-
- 25
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
An experimental test of the effects of redacting grant applicant identifiers on peer review outcomes
eLife 10:e71368.
https://doi.org/10.7554/eLife.71368
Further reading
-
The study of science itself is a growing field of research.
-
- Cancer Biology
- Cell Biology
Testicular microcalcifications consist of hydroxyapatite and have been associated with an increased risk of testicular germ cell tumors (TGCTs) but are also found in benign cases such as loss-of-function variants in the phosphate transporter SLC34A2. Here, we show that fibroblast growth factor 23 (FGF23), a regulator of phosphate homeostasis, is expressed in testicular germ cell neoplasia in situ (GCNIS), embryonal carcinoma (EC), and human embryonic stem cells. FGF23 is not glycosylated in TGCTs and therefore cleaved into a C-terminal fragment which competitively antagonizes full-length FGF23. Here, Fgf23 knockout mice presented with marked calcifications in the epididymis, spermatogenic arrest, and focally germ cells expressing the osteoblast marker Osteocalcin (gene name: Bglap, protein name). Moreover, the frequent testicular microcalcifications in mice with no functional androgen receptor and lack of circulating gonadotropins are associated with lower Slc34a2 and higher Bglap/Slc34a1 (protein name: NPT2a) expression compared with wild-type mice. In accordance, human testicular specimens with microcalcifications also have lower SLC34A2 and a subpopulation of germ cells express phosphate transporter NPT2a, Osteocalcin, and RUNX2 highlighting aberrant local phosphate handling and expression of bone-specific proteins. Mineral disturbance in vitro using calcium or phosphate treatment induced deposition of calcium phosphate in a spermatogonial cell line and this effect was fully rescued by the mineralization inhibitor pyrophosphate. In conclusion, testicular microcalcifications arise secondary to local alterations in mineral homeostasis, which in combination with impaired Sertoli cell function and reduced levels of mineralization inhibitors due to high alkaline phosphatase activity in GCNIS and TGCTs facilitate osteogenic-like differentiation of testicular cells and deposition of hydroxyapatite.
