Studies of both males and females are essential to the advancement of human health, and the influences of sex on the prevalence, presentation, and progression of many disease states is profound. Yet, within the biological sciences, it has been a common and preferential practice to utilize male research subjects in basic and pre-clinical research (Beery and Zucker, 2011; Kong et al., 2016; Sugimoto et al., 2019; Yoon et al., 2014). This male bias stems from the misconception that female animals increase experimental variability due to cyclical fluctuating hormones and the historical belief that no major differences exist between the sexes outside of reproductive functions (Institute of Medicine, 2001). These biases are not limited to the basic sciences, but extend into clinical research as well (Geller et al., 2018; Mansukhani et al., 2016; Prakash et al., 2018; Scott et al., 2018).

Initial reports calling for the inclusion of females in research and which describe the limitations of sex-biased studies began in the 1990s and extended in to the early 2000s (Berkley, 1992; Holdcroft, 2007; Mogil and Chanda, 2005). In 2009, Beery and Zucker conducted a multi-disciplinary review of primary literature which quantified the extent of sex-bias across several research areas in the biological sciences (Beery and Zucker, 2011). Since that report, there have been numerous calls to address this issue through sex-inclusive research practices and policies (Kim et al., 2010; Klein et al., 2015; Mazure and Jones, 2015; Woodruff, 2014), culminating in 2016 when the National Institutes of Health (NIH) in the United States implemented a policy requiring investigators to consider sex as a biological variable (Clayton and Collins, 2014). The intent of the policy is to ensure equal representation of males and females in vertebrate research studies, unless there is significant justification to support the use of a single-sex. Many lauded the policy (Mogil, 2016; Shansky and Woolley, 2016), yet there were still those who saw it as unnecessary and feared that it would be time consuming, costly, increase experimental variability, and require expertise in the study of sex differences (Woitowich and Woodruff, 2019). Considering sex as a biological variable does not require investigators to power studies in order to determine sex differences nor does it ask investigators to analyze data by sex. Yet, these common misconceptions persist, despite clarifications and guidance surrounding sex-inclusive research practices (Arnegard et al., 2020; Becker et al., 2016; Clayton, 2018; Miller et al., 2017; Shansky, 2019). Recently, several studies have monitored the progress of sex-inclusive research practices following NIH policy implementation in the fields of microbiology and immunology (Potluri et al., 2017), as well as neuroscience (Mamlouk et al., 2020) utilizing methodologies similar to Beery and Zucker. Here, we present a 10 year follow-up study to the initial Beery and Zucker report by conducting a systematic review to assess sex-inclusive research practices within nine of the biological disciplines and 34 of the scholarly journals originally surveyed in 2009. We provide an updated perspective on the state of sex-inclusive research within the biological sciences, and highlight areas of improvement alongside shortcomings in the decade since Beery and Zucker conducted their original study.

In 2009, Beery and Zucker conducted a bibliometric analysis of 841 articles from high-impact journals, across ten biological disciplines which quantified the extent of male-bias in research and a noted lack of sex-based analyses when males and females were both included as research subjects (Beery and Zucker, 2011). We recapitulated the work of Beery and Zucker utilizing a similar bibliometric analysis of 720 journal articles, corresponding to nine of the original disciplines and 34 journals surveyed in 2009 (Table 1).

Subject sex across disciplines

In 2019, 49% (n = 356) of studies reported using both male and female research subjects, resulting in a significant increase in sex inclusion demographics compared to 28% of articles surveyed 2009 (n = 232, p<0.0001; Figure 1A). Six of the nine disciplines demonstrated a significant increase in the use of both sexes (Figure 1). Between 2009 and 2019, the largest increases in sex-inclusive studies were seen in the fields of neuroscience (29% vs. 63%, p<0.0001) and immunology (16% vs. 46%, p<0.0001), followed by endocrinology (30% vs. 56%, p=0.001), general biology (34% vs. 59%, p=0.002), physiology (13% vs. 36%, p=0.001), and behavioral physiology (43% vs. 61%, p=0.018). In reproduction, single-sex studies remained the norm, and studies of both males and females increased only marginally (10% vs. 14%, p=0.35), while the number of female only research studies increased, corresponding to a female to male subject ratio of 1.6:1 in 2009 to 3.6:1 in 2019. Behavior remained the most inclusive biological discipline with 70% and 81% of studies reporting the use of both sexes in 2009 and 2019, respectively, largely driven by sex-inclusive field studies. Pharmacology was the only field to trend downward with 29% of articles reporting the use of both sexes in 2019 compared to 33% in 2009 (p=0.607). Likewise, there was an increase in the male to female subject ratio from 5:1 in 2009 to 5.8:1 in 2019.

Figure 1

Download asset Open asset

Description of sample size by sex across disciplines

For articles that reported the inclusion of both sexes in 2019, data were collected on whether the authors provided a description of the sample size (n) by sex. Out of the 356 articles that used both sexes, 27% failed to provide a description of the sample size by sex (Figure 2A). Neuroscience articles failed to provide a description of the sample size by sex 52% (n = 26) of the time, along with general biology at 47% (n = 22) and immunology at 43% (n = 19).

Figure 2

Download asset Open asset

Notably, the number of sex-inclusive research studies has significantly increased across most biological disciplines. At face value, this change is encouraging and suggests that the scientific community may have an increased awareness and understanding of the need for sex-inclusive research and its contribution to experimental rigor and reproducibility (Clayton, 2018; Miller et al., 2017). At the same time, close to one third of all research studies that utilized both male and female subjects failed to quantify their sample size by sex. Ironically, this is most prevalent in the fields which reported the greatest increases in sex-inclusive research (ex. neuroscience, immunology, and general biology) At best, this result indicates that investigators may not think it is important to provide a description of the sample size by sex in the absence of sex-based analyses. In a less ideal case, the representation of males and females is not well balanced, and this may be intentionally obscured. Single-sex studies are valid and warranted, provided there is evidence-based rationale for the case. Yet, several studies explicitly stated that they excluded both sexes as a means to prevent experimental variability, which is an erroneous belief and unsound research practice (Beery, 2018; Prendergast et al., 2014).

Perhaps most concerning, improvements in the inclusion of both sexes over the past decade have not been accompanied by general improvement in sex-based analyses, despite repeated calls and guidelines for such analyses (Beltz et al., 2019; Clayton, 2018; Clayton and Collins, 2014; Hankivsky et al., 2018; Prager, 2017). Sex-based analyses may uncover sex differences for a given trait, prompting the development of sex-specific prevention strategies, drug targets, or other therapies beneficial to both sexes (Yang et al., 2019). And while it is reasonable to aggregate and analyze data from both sexes if it has been established that there are no sex-differences for a given trait or condition, out of the 720 articles reviewed here, only two conveyed this information in their methods. When this information is lacking, the reader is tasked with making the assumption that either there are no sex differences or that sex-differences have yet to be examined. In either case, this can lead to redundant research efforts requiring additional time, money, and biological resources.

The data presented here highlight a continued need for education, awareness, and advocacy surrounding sex-based research practices including the consideration of sex as a biological variable. We call upon academic publishers to require a description of sex, rationale for single-sex studies or lack of sex-based analyses in the experimental methods. In the absence of formal policies, reviewers can ask for these essential criteria. In addition, funders can also contribute to the advancement of rigorous sex-inclusive science by requiring grant proposals to include appropriate sex-based reporting and analyses and determine funding success on the evaluation of sex and other key biological variables. Lastly, we call upon universities to encourage the consideration of sex as a biological variable through institutional review boards (IRBs) and institutional animal care and use committees (IACUC) oversight (Duffy et al., 2020) and by providing instruction to biomedical trainees on sex-inclusion, reporting, and analyses through established responsible conduct of research modules and within medical school curricula. Only together, through concerted, tripartite efforts at the institutional, funder, and publishing levels will the consideration of sex as a biological variable become standard practice (Tannenbaum et al., 2019). Together, this will allow us to improve our understanding of health and disease for both men and women and to further the reality of personalized medicine.

A systematic sampling of journal articles from 2019 was conducted using the methodologies originally described in Beery and Zucker, 2011. All articles were reviewed and coded by one of us (NCW) in order to minimize coding bias. Briefly, journal articles were assessed for sex-inclusive research practices from nine biological disciplines and 34 journals sampled by Beery and Zucker, 2011. These disciplines included: General Biology, Immunology, Neuroscience, Physiology, Pharmacology, Reproduction, Endocrinology, Behavioral Physiology, and Behavior. Zoology, which was studied by Beery and Zucker, was excluded here due to a limited number of mammalian studies available to survey at the time of manuscript preparation. Four journals were selected to represent each discipline, with the exception of Reproduction (Table 1). For each journal, the first 20 primary research articles which met eligibility criteria were surveyed in 2019. For the two reproductive biology journals, the first 40 journal articles were surveyed for 2019. For the majority of disciplines, the first 20 research articles which met eligibility criteria were published between January and April of 2019, whereas articles from other disciplines were published between January through June (Endocrinology), August (Behavioral Physiology) and October (Behavior) of 2019.

The eligibility criteria for studies in this analysis were as follows.

Inclusion criteria (all criteria required): i) Reported use of any vertebrate mammal in some part of the experimental methods, including those which describe the generation of primary cell culture; ii) Published after January 1 st, 2019; iii) Published in the English language.

Exclusion criteria (each criterion can exclude): i) Type of article: review articles, brief communications, or viewpoints; ii) Articles published in a special or themed issue; iii) Reports utilizing fetal organisms or those restricted to immortal cell lines.

When journals were arranged by subtopics, articles were sampled evenly across several topics. In journals such as Nature and Science, only articles pertaining to the biological sciences were considered.

Articles were coded for sex. Sex was recorded as male, female, both sexes, or unspecified. Following the strategy of Beery and Zucker, 2011, coding was biased in favor of inclusivity and articles were categorized as using both sexes when different parts of a study utilized different sexes. Likewise, field studies were categorized as investigating both sexes when this was explicitly noted or could be inferred by the methods provided. Articles which utilized both sexes were further evaluated for a description of the sample size by sex and whether data were analyzed by sex, including sex as a covariate or subgroup analyses by sex. For all articles reviewed, we noted if the authors provided rationale for the use of a single sex or the lack of sex-based analyses.

Data analyses were primarily qualitative, with a small quantitative component. Descriptive statistics were used where appropriate. Nominal data were described as n (%). We compared the 2019 data to 2009 data in Beery and Zucker, 2011. Chi-squared tests were used to assess differences between the use of both sexes in 2009 compared to 2019, and the number of studies which analyzed data by sex in 2009 compared to 2019 (GraphPad Prism, version 7.0). p-values<0.05 were considered significant.

All data generated or analysed during this study are included in the manuscript and supporting files. Source data files have been provided for Figures 1 and 2.

1. 1. Arnegard ME
   2. Whitten LA
   3. Hunter C
   4. Clayton JA

   (2020) Sex as a biological variable: a 5-year progress report and call to action

   Journal of Women's Health.

   https://doi.org/10.1089/jwh.2019.8247

   • PubMed
   • Google Scholar
2. 1. Becker JB
   2. Prendergast BJ
   3. Liang JW

   (2016) Female rats are not more variable than male rats: a meta-analysis of neuroscience studies

   Biology of Sex Differences 7:34.

   https://doi.org/10.1186/s13293-016-0087-5

   • PubMed
   • Google Scholar
3. 1. Beery AK

   (2018) Inclusion of females does not increase variability in rodent research studies

   Current Opinion in Behavioral Sciences 23:143–149.

   https://doi.org/10.1016/j.cobeha.2018.06.016

   • Google Scholar
4. 1. Beery AK
   2. Zucker I

   (2011) Sex bias in neuroscience and biomedical research

   Neuroscience & Biobehavioral Reviews 35:565–572.

   https://doi.org/10.1016/j.neubiorev.2010.07.002

   • PubMed
   • Google Scholar
5. 1. Beltz AM
   2. Beery AK
   3. Becker JB

   (2019) Analysis of sex differences in pre-clinical and clinical data sets

   Neuropsychopharmacology 44:2155–2158.

   https://doi.org/10.1038/s41386-019-0524-3

   • PubMed
   • Google Scholar
6. 1. Berkley KJ

   (1992) Vive la différence!

   Trends in Neurosciences 15:331–332.

   https://doi.org/10.1016/0166-2236(92)90048-D

   • Google Scholar
7. 1. Clayton JA

   (2018) Applying the new SABV (sex as a biological variable) policy to research and clinical care

   Physiology & Behavior 187:2–5.

   https://doi.org/10.1016/j.physbeh.2017.08.012

   • Google Scholar
8. 1. Clayton JA
   2. Collins FS

   (2014) NIH to balance sex in cell and animal studies

   Nature 509:282–283.

   https://doi.org/10.1038/509282a

   • PubMed
   • Google Scholar
9. 1. Duffy KA
   2. Ziolek TA
   3. Epperson CN

   (2020) Filling the regulatory gap: potential role of institutional review boards in promoting consideration of sex as a biological variable

   Journal of Women's Health.

   https://doi.org/10.1089/jwh.2019.8084

   • PubMed
   • Google Scholar
10. 1. Geller SE
    2. Koch AR
    3. Roesch P
    4. Filut A
    5. Hallgren E
    6. Carnes M

    (2018) The more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials

    Academic Medicine : Journal of the Association of American Medical Colleges 93:630–635.

    https://doi.org/10.1097/ACM.0000000000002027

    • PubMed
    • Google Scholar
11. 1. Hankivsky O
    2. Springer KW
    3. Hunting G

    (2018) Beyond sex and gender difference in funding and reporting of health research

    Research Integrity and Peer Review 3:6.

    https://doi.org/10.1186/s41073-018-0050-6

    • PubMed
    • Google Scholar
12. 1. Holdcroft A

    (2007) Gender bias in research: how does it affect evidence based medicine?

    Journal of the Royal Society of Medicine 100:2–3.

    https://doi.org/10.1177/014107680710000102

    • PubMed
    • Google Scholar
13. Book

    1. Institute of Medicine

    (2001) Exploring the Biological Contributions to Human Health: Does Sex Matter?

    Washington, DC:  National Academies Press.

    https://doi.org/10.1089/152460901300233902

    • Google Scholar
14. 1. Kim AM
    2. Tingen CM
    3. Woodruff TK

    (2010) Sex bias in trials and treatment must end

    Nature 465:688–689.

    https://doi.org/10.1038/465688a

    • PubMed
    • Google Scholar
15. 1. Klein SL
    2. Schiebinger L
    3. Stefanick ML
    4. Cahill L
    5. Danska J
    6. de Vries GJ
    7. Kibbe MR
    8. McCarthy MM
    9. Mogil JS
    10. Woodruff TK
    11. Zucker I

    (2015) Sex inclusion in basic research drives discovery

    PNAS 112:5257–5258.

    https://doi.org/10.1073/pnas.1502843112

    • PubMed
    • Google Scholar
16. 1. Kong BY
    2. Haugh IM
    3. Schlosser BJ
    4. Getsios S
    5. Paller AS

    (2016) Mind the gap: sex bias in basic skin research

    Journal of Investigative Dermatology 136:12–14.

    https://doi.org/10.1038/JID.2015.298

    • PubMed
    • Google Scholar
17. 1. Mamlouk GM
    2. Dorris DM
    3. Barrett LR
    4. Meitzen J

    (2020) Sex bias and omission in neuroscience research is influenced by research model and journal, but not reported NIH funding

    Frontiers in Neuroendocrinology 57:100835.

    https://doi.org/10.1016/j.yfrne.2020.100835

    • PubMed
    • Google Scholar
18. 1. Mansukhani NA
    2. Yoon DY
    3. Teter KA
    4. Stubbs VC
    5. Helenowski IB
    6. Woodruff TK
    7. Kibbe MR

    (2016) Determining if sex bias exists in human surgical clinical research

    JAMA Surgery 151:1022–1030.

    https://doi.org/10.1001/jamasurg.2016.2032

    • PubMed
    • Google Scholar
19. 1. Mazure CM
    2. Jones DP

    (2015) Twenty years and still counting: including women as participants and studying sex and gender in biomedical research

    BMC Women's Health 15:94.

    https://doi.org/10.1186/s12905-015-0251-9

    • PubMed
    • Google Scholar
20. 1. Miller LR
    2. Marks C
    3. Becker JB
    4. Hurn PD
    5. Chen WJ
    6. Woodruff T
    7. McCarthy MM
    8. Sohrabji F
    9. Schiebinger L
    10. Wetherington CL
    11. Makris S
    12. Arnold AP
    13. Einstein G
    14. Miller VM
    15. Sandberg K
    16. Maier S
    17. Cornelison TL
    18. Clayton JA

    (2017) Considering sex as a biological variable in preclinical research

    The FASEB Journal 31:29–34.

    https://doi.org/10.1096/fj.201600781r

    • PubMed
    • Google Scholar
21. 1. Mogil JS

    (2016) Equality need not be painful

    Nature 535:S7.

    https://doi.org/10.1038/535S7a

    • Google Scholar
22. 1. Mogil JS
    2. Chanda ML

    (2005) The case for the inclusion of female subjects in basic science studies of pain

    Pain 117:1–5.

    https://doi.org/10.1016/j.pain.2005.06.020

    • PubMed
    • Google Scholar
23. 1. Potluri T
    2. Engle K
    3. Fink AL
    4. Vom Steeg LG
    5. Klein SL

    (2017) Sex reporting in preclinical microbiological and immunological research

    mBio 8:e01868-17.

    https://doi.org/10.1128/mBio.01868-17

    • PubMed
    • Google Scholar
24. 1. Prager EM

    (2017) Addressing sex as a biological variable

    Journal of Neuroscience Research 95:11.

    https://doi.org/10.1002/jnr.23979

    • PubMed
    • Google Scholar
25. 1. Prakash VS
    2. Mansukhani NA
    3. Helenowski IB
    4. Woodruff TK
    5. Kibbe MR

    (2018) Sex bias in interventional clinical trials

    Journal of Women's Health 27:1342–1348.

    https://doi.org/10.1089/jwh.2017.6873

    • PubMed
    • Google Scholar
26. 1. Prendergast BJ
    2. Onishi KG
    3. Zucker I

    (2014) Female mice liberated for inclusion in neuroscience and biomedical research

    Neuroscience & Biobehavioral Reviews 40:1–5.

    https://doi.org/10.1016/j.neubiorev.2014.01.001

    • Google Scholar
27. 1. Scott PE
    2. Unger EF
    3. Jenkins MR
    4. Southworth MR
    5. McDowell TY
    6. Geller RJ
    7. Elahi M
    8. Temple RJ
    9. Woodcock J

    (2018) Participation of women in clinical trials supporting FDA approval of cardiovascular drugs

    Journal of the American College of Cardiology 71:1960–1969.

    https://doi.org/10.1016/j.jacc.2018.02.070

    • PubMed
    • Google Scholar
28. 1. Shansky RM

    (2019) Are hormones a "female problem" for animal research?

    Science 364:825–826.

    https://doi.org/10.1126/science.aaw7570

    • PubMed
    • Google Scholar
29. 1. Shansky RM
    2. Woolley CS

    (2016) Considering sex as a biological variable will be valuable for neuroscience research

    The Journal of Neuroscience 36:11817–11822.

    https://doi.org/10.1523/JNEUROSCI.1390-16.2016

    • PubMed
    • Google Scholar
30. 1. Sugimoto CR
    2. Ahn Y-Y
    3. Smith E
    4. Macaluso B
    5. Larivière V

    (2019) Factors affecting sex-related reporting in medical research: a cross-disciplinary bibliometric analysis

    The Lancet 393:550–559.

    https://doi.org/10.1016/S0140-6736(18)32995-7

    • Google Scholar
31. 1. Tannenbaum C
    2. Ellis RP
    3. Eyssel F
    4. Zou J
    5. Schiebinger L

    (2019) Sex and gender analysis improves science and engineering

    Nature 575:137–146.

    https://doi.org/10.1038/s41586-019-1657-6

    • PubMed
    • Google Scholar
32. 1. Woitowich NC
    2. Woodruff TK

    (2019) Implementation of the NIH sex-inclusion policy: attitudes and opinions of study section members

    Journal of Women's Health 28:9–16.

    https://doi.org/10.1089/jwh.2018.7396

    • PubMed
    • Google Scholar
33. 1. Woodruff TK

    (2014) Sex, equity, and science

    PNAS 111:5063–5064.

    https://doi.org/10.1073/pnas.1404203111

    • PubMed
    • Google Scholar
34. 1. Yang W
    2. Warrington NM
    3. Taylor SJ
    4. Whitmire P
    5. Carrasco E
    6. Singleton KW
    7. Wu N
    8. Lathia JD
    9. Berens ME
    10. Kim AH
    11. Barnholtz-Sloan JS
    12. Swanson KR
    13. Luo J
    14. Rubin JB

    (2019) Sex differences in GBM revealed by analysis of patient imaging, transcriptome, and survival data

    Science Translational Medicine 11:eaao5253.

    https://doi.org/10.1126/scitranslmed.aao5253

    • Google Scholar
35. 1. Yoon DY
    2. Mansukhani NA
    3. Stubbs VC
    4. Helenowski IB
    5. Woodruff TK
    6. Kibbe MR

    (2014) Sex bias exists in basic science and translational surgical research

    Surgery 156:508–516.

    https://doi.org/10.1016/j.surg.2014.07.001

    • PubMed
    • Google Scholar

Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor (Cassidy Sugimoto) and the eLife Features Editor (Peter Rodgers). The following individuals involved in review of your submission have agreed to reveal their identity: Rebecca Shansky (Reviewer #2); Londa Schiebinger (Reviewer #3).

The reviewers and editors have discussed the reviews and we have drafted this decision to help you prepare a revised submission. We hope you will be able to submit the revised version within two months.

Summary:

This manuscript is a 10-year check in on a seminal paper (Beery & Zucker 2011) that systematically examined sex bias in biomedical animal research. Given the introduction of NIH's "Considering Sex as a Biological Variable" (SABV) mandate in 2016, it is important to assess whether research practices have changed over the past decade.

The authors conducted a survey of 2019 publications in ten major fields, and report that in some fields, inclusion of both sexes has increased but that for the most part this has not been accompanied by an increase in analysis of data by sex or reporting the n's of each sex used. The authors call for journals to require more detailed information about the sex of animals and justification of single sex studies. I especially appreciate the assessment of "excuses" for the single sex studies.

This study is a critical addition to the literature and will help biomedical scientists understand how well SABV is working and what needs to be done to make it even more successful.

This is an important article and it should be published once the following points have been addressed.

Essential revisions:

1) Introduction and Discussion: It was disappointing to see that the introduction and discussion did not address/include several key historical and recently published studies. These studies include some of the first studies that attempted to raise awareness regarding the lack of sex reporting (Berkley 1992; Mogil and Chanda 2005; both published before the initial 2009 study), updates to the 2009 study that robustly analyzed key fields that were documented as problematic in the 2009 study (Potluri et al., 2017; Mamlouk et al., 2020), and recently published arguments and analyses that are advancing sex-inclusive research (Shanskey, 2019; Galea et al., 2020). There are others as well (eg, Becker et al., 2016). While I am well aware that not every relevant paper can be cited/discussed, I encourage the authors to enhance the robustness of their literature search to help place their important results within the broader context.

2) I would encourage the authors to point out that SABV explicitly states that experiments need not be powered to detect sex differences or that data be analyzed by sex. However, their calls to require reporting of sex n's are well founded and will improve rigor and reproducibility.

3) The authors call upon academic publishers to require sex reporting. They should also call upon funders to require appropriate sex reporting and analysis in grant proposals--and make funding decision on the quality of that (and other) analysis. The authors should also call upon universities (medical schools) to teach sex reporting and analysis in the curriculum. In Tannenbaum et al (Nature 575: 137-146) the present reviewer [LS] and colleagues discuss the three pillars of the science infrastructure: Funding agencies, Universities, and Peer-reviewed journals. I encourage the authors here to add funders and universities to the paragraph where they discuss academic publishers.

4) Methods: "First 20 primary research articles" - Since only the first 20 articles were analyzed does this mean that most of the data was collected in studies published between January-March 2019? It would be a good idea to document this in the methods in case the data from 2019 need to be compared to future analyses that select across the entire year.

5) Methods: Interrater reliability: Did more than one person analyze studies? Were intra and inter-rater reliability controls performed to ensure accurate analysis?

6) Methods: "Organismal mammalian work" - This label is vague and could be divergently interpreted by different readers. Does this just mean whole organism behavioral analysis? Does it include all mammals, including primates and humans, or just rodents? Beyond answering these specific questions, the entire a priori article section criteria needs to be much better documented. This documentation should include how the employed protocol may potentially skew findings.

7) Methods: "Whether data were analyzed be sex." This definition is vague and requires further documentation. What analyses were considered an analysis by sex? For example, if sex was employed as a covariate, was this considered a sex analysis? What if the data or analysis/statistics were not shown? What if data were presented as disaggregated by sex but not further analyzed? Similar to the previous point, this section requires more robust documentation so that the reader can understand how this study was conducted.

[We repeat the reviewers’ points here in italic, and include our replies in Roman.]

Essential revisions:

1) Introduction and Discussion: It was disappointing to see that the introduction and discussion did not address/include several key historical and recently published studies. These studies include some of the first studies that attempted to raise awareness regarding the lack of sex reporting (Berkley 1992; Mogil and Chanda 2005; both published before the initial 2009 study), updates to the 2009 study that robustly analyzed key fields that were documented as problematic in the 2009 study (Potluri et al., 2017; Mamlouk et al., 2020), and recently published arguments and analyses that are advancing sex-inclusive research (Shanskey, 2019; Galea et al., 2020). There are others as well (eg, Becker et al., 2016). While I am well aware that not every relevant paper can be cited/discussed, I encourage the authors to enhance the robustness of their literature search to help place their important results within the broader context.

We thank the reviewers for these suggestions and have enhanced our introduction to include a robust review of the literature related to the SABV policy and sex- and gender-inclusive research practices. Which, in turn, highlights the importance of this work by including additional historical context.

2) I would encourage the authors to point out that SABV explicitly states that experiments need not be powered to detect sex differences or that data be analyzed by sex. However, their calls to require reporting of sex n's are well founded and will improve rigor and reproducibility.

We have modified our introduction to ensure the clarification of this common misconception is included and provide more detailed information about the SABV policy.

3) The authors call upon academic publishers to require sex reporting. They should also call upon funders to require appropriate sex reporting and analysis in grant proposals--and make funding decision on the quality of that (and other) analysis. The authors should also call upon universities (medical schools) to teach sex reporting and analysis in the curriculum. In Tannenbaum et al (Nature 575: 137-146) the present reviewer [LS] and colleagues discuss the three pillars of the science infrastructure: Funding agencies, Universities, and Peer-reviewed journals. I encourage the authors here to add funders and universities to the paragraph where they discuss academic publishers.

We have added these additional “calls to action” and included recent work encouraging IRB and IACUC oversight as well (Duffy et al 2020).

4) Methods: "First 20 primary research articles" - Since only the first 20 articles were analyzed does this mean that most of the data was collected in studies published between January-March 2019? It would be a good idea to document this in the methods in case the data from 2019 need to be compared to future analyses that select across the entire year.

We thank the reviewers for this comment and have added this information into the Methods section. For the majority of fields, the articles analyzed here were published between January - April 2019, but some fields extended further into the calendar year, which we noted in the Methods.

5) Methods: Interrater reliability: Did more than one person analyze studies? Were intra and inter-rater reliability controls performed to ensure accurate analysis?

All studies were analyzed by the corresponding author. We have modified the Methods section to reflect this information.

6) Methods: "Organismal mammalian work" - This label is vague and could be divergently interpreted by different readers. Does this just mean whole organism behavioral analysis? Does it include all mammals, including primates and humans, or just rodents? Beyond answering these specific questions, the entire a priori article section criteria needs to be much better documented. This documentation should include how the employed protocol may potentially skew findings.

We updated the Methods section to include a detailed listing of the eligibility criteria for each article included in this study.

7) Methods: "Whether data were analyzed be sex." This definition is vague and requires further documentation. What analyses were considered an analysis by sex? For example, if sex was employed as a covariate, was this considered a sex analysis? What if the data or analysis/statistics were not shown? What if data were presented as disaggregated by sex but not further analyzed? Similar to the previous point, this section requires more robust documentation so that the reader can understand how this study was conducted.

We thank the authors for this comment and have updated the Methods section to reflect a more accurate description of sex-based analyses.

Author details

1. Nicole C Woitowich

   Nicole C Woitowich is in the Women’s Health Research Institute and the Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, United States

   Contribution

   Conceptualization, Data curation, Formal analysis, Supervision, Validation, Investigation, Visualization, Methodology, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   nicole.woitowich@northwestern.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-3449-2547

2. Annaliese Beery

   Annaliese K Beery is in the Department of Psychology, the Department of Biology, and the Program in Neuroscience, Smith College, Northampton, United States

   Contribution

   Conceptualization, Data curation, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1249-9182

3. Teresa Woodruff

   Teresa K Woodruff is in the Women’s Health Research Institute and the Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, United States

   Contribution

   Conceptualization, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1197-3399

• 8,573

  views

• 775

  downloads

• 211

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.