Meta-Research: Journal policies and editors’ opinions on peer review
- Interdisciplinary Metaresearch Group, School of BioSciences, University of Melbourne, Australia
- Department of Educational Sciences, University of Groningen, Netherlands
- School of Historical and Philosophical Studies, University of Melbourne, Australia
Figures
Figure 1 with 2 supplements
Participating editors’ in principle stances on the six topics raised in Survey B.
The figures presented are limited to statements that provided a clear view for or against the topic of interest. An interactive version of this figure reporting results by discipline can be viewed at https://plotly.com/~dghamilton/9/ (Supplementary file 1). Source data for the figure can be found at https://doi.org/10.17605/osf.io/cy2re.
Figure 1—figure supplement 1
Survey response rate by discipline.
Figure 1—figure supplement 2
Distribution of impact factors among invited and participating journals by discipline.
Tables
Table 1
Pre-review policies for all journals and by discipline.
| All journals | Ecology | Psychology | Economics | Medicine | Physics | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | N | % | N | % | |
| Plagiarism software usage (N=317) | ||||||||||||
| Never | 7 | 2 | 4 | 4 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 4 |
| Always | 154 | 49 | 46 | 51 | 45 | 54 | 24 | 31 | 25 | 64 | 14 | 54 |
| If suspicion has been raised | 84 | 26 | 24 | 27 | 19 | 23 | 30 | 38 | 4 | 10 | 7 | 27 |
| At editor's discretion | 54 | 17 | 14 | 16 | 13 | 15 | 18 | 23 | 5 | 13 | 4 | 15 |
| I don't know | 5 | 2 | 0 | 0 | 3 | 4 | 2 | 3 | 0 | 0 | 0 | 0 |
| Other | 13 | 4 | 2 | 2 | 3 | 4 | 3 | 4 | 5 | 13 | 0 | 0 |
| Recommending reviewers (N=321) | ||||||||||||
| No | 73 | 23 | 0 | 0 | 18 | 21 | 47 | 59 | 4 | 10 | 4 | 14 |
| Yes - Recommend for only | 27 | 8 | 11 | 12 | 6 | 7 | 2 | 3 | 4 | 10 | 4 | 14 |
| Yes - Recommend against only | 12 | 4 | 3 | 3 | 5 | 6 | 3 | 4 | 1 | 2 | 0 | 0 |
| Yes - Recommend for and against | 197 | 61 | 75 | 83 | 51 | 61 | 23 | 29 | 29 | 72 | 19 | 68 |
| Other | 12 | 4 | 1 | 1 | 4 | 5 | 4 | 5 | 2 | 5 | 1 | 4 |
| Outsourcing peer review (N=318) | ||||||||||||
| No | 315 | 99 | 90 | 100 | 82 | 100 | 78 | 100 | 38 | 95 | 27 | 96 |
| Yes | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 1 | 4 |
| Other | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 |
Table 2
Blinding policies for all journals and by discipline.
| All journals | Ecology | Psychology | Economics | Medicine | Physics | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | N | % | N | % | |
| Open identities | 3 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 2 | 5 | 0 | 0 |
| Single-blind | 176 | 57 | 68 | 78 | 16 | 20 | 33 | 43 | 33 | 87 | 26 | 100 |
| Single-blind (hybrid) | 12 | 4 | 3 | 3 | 8 | 10 | 1 | 1 | 0 | 0 | 0 | 0 |
| Double-blind (hybrid) | 4 | 1 | 1 | 1 | 3 | 4 | 0 | 0 | 0 | 0 | 0 | 0 |
| Double-blind | 109 | 36 | 15 | 17 | 51 | 65 | 40 | 52 | 3 | 8 | 0 | 0 |
| Triple-blind | 3 | 1 | 0 | 0 | 1 | 1 | 2 | 3 | 0 | 0 | 0 | 0 |
Table 3
Situations where an editor may edit a reviewer’s report.
| Never acceptable to edit the report | Acceptable to edit without reviewer's permission | Acceptable to edit, but only with reviewer's permission | ||||
|---|---|---|---|---|---|---|
| N | % | N | % | N | % | |
| When a reviewer identifies themselves in a blinded peer review framework (N=276) | 93 | 34 | 109 | 39 | 74 | 27 |
| When a reviewer has used inappropriate or offensive language (N=291) | 44 | 15 | 170 | 58 | 77 | 26 |
| When the reviewer has made an inappropriate reference to an author's gender, age etc (N=290) | 48 | 17 | 163 | 56 | 79 | 27 |
| When there are spelling and/or grammatical errors in the review (N=294) | 104 | 35 | 141 | 48 | 49 | 17 |
| When the review has English language problems (N=292) | 95 | 33 | 124 | 42 | 73 | 25 |
| When the reviewer has left in their comments to the editor (N-290) | 50 | 17 | 179 | 62 | 61 | 21 |
| When the editor disagrees with the reviewer's recommendation (N=293) | 238 | 81 | 22 | 8 | 33 | 11 |
Table 4
Journal policies on the sharing of research data, materials and code.
| Research data (N=294) | Research materials (N=264) | Research code (N=255) | ||||
|---|---|---|---|---|---|---|
| N | % | N | % | N | % | |
| Encourages sharing but it is not required | 168 | 57 | 143 | 54 | 133 | 52 |
| Must make available if requested | 41 | 14 | 29 | 11 | 32 | 13 |
| In-text statement required | 52 | 18 | 34 | 13 | 29 | 11 |
| Requires posting to a trusted repository | 34 | 12 | 16 | 6 | 19 | 7 |
| No policy | 65 | 22 | 69 | 26 | 65 | 25 |
| Not applicable | 10 | 3 | 13 | 5 | 11 | 4 |
| I don't know | 4 | 1 | 4 | 2 | 6 | 2 |
| Other | 8 | 3 | 7 | 3 | 7 | 3 |
-
Percentages do not add up to 100% due to multiple answers being possible
Additional files
-
Supplementary file 1
Interactive version of Figure 1.
- https://cdn.elifesciences.org/articles/62529/elife-62529-supp1-v2.zip
-
Supplementary file 2
Categories used to define disciplines in InCites; codebook reliability testing.
- https://cdn.elifesciences.org/articles/62529/elife-62529-supp2-v2.docx
-
Supplementary file 3
Survey questions (A and B).
- https://cdn.elifesciences.org/articles/62529/elife-62529-supp3-v2.docx
-
Supplementary file 4
Responses to questions in survey A.
- https://cdn.elifesciences.org/articles/62529/elife-62529-supp4-v2.docx
-
Transparent reporting form
- https://cdn.elifesciences.org/articles/62529/elife-62529-transrepform-v2.pdf
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Meta-Research: Journal policies and editors’ opinions on peer review
eLife 9:e62529.
https://doi.org/10.7554/eLife.62529
