Reporting science clearly and accurately is a fundamental part of the scientific process, facilitating both the dissemination of knowledge and the reproducibility of results. The clarity of written language can be quantified using readability formulas, which estimate how understandable written texts are (Flesch, 1948; Kincaid et al., 1975; Chall and Dale, 1995; Danielson, 1987; DuBay, 2004; Štajner et al., 2012). Texts written at different times can vary in their readability: trends towards simpler language have been observed in US presidential speeches (Lim, 2008), novels (Danielson et al., 1992; Jatowt and Tanaka, 2012) and news articles (Stevenson, 1964). There are studies that have investigated linguistic trends within the scientific literature. One study showed an increase in positive sentiment (Vinkers et al., 2015), finding that positive words such as 'novel' have increased dramatically in scientific texts since the 1970s. A tentative increase in complexity has been reported in scientific texts in a limited dataset (Hayes, 1992), but the extent of this phenomenon and any underlying reasons for such a trend remain unknown.

To investigate trends in scientific readability over time, we downloaded 709,577 article abstracts from PubMed, from 123 highly cited journals selected from 12 fields of research (Figure 1A–C). These journals cover general, biomedical and life sciences. This journal list included, among others, Nature, Science, NEJM, The Lancet, PNAS and JAMA (see Materials and methods and Supplementary file 1) and the publication dates ranged from 1881 to 2015. We quantified the reading level of each abstract using two established measures of readability: the Flesch Reading Ease (FRE; Flesch, 1948; Kincaid et al., 1975) and the New Dale-Chall Readability Formula (NDC; Chall and Dale, 1995). The FRE is calculated using the number of syllables per word and the number of words in each sentence. The NDC is calculated using the number of words in each sentence and the percentage of 'difficult words'. Difficult words are defined as those words which do not belong to a predefined list of common words (see Materials and methods). Lower readability is indicated by a low FRE score or a high NDC score (Figure 1A).

Figure 1

Download asset Open asset

The primary research question was to examine how the readability of an article's abstract relates to its year of publication. We observed a strong decreasing trend of the average yearly FRE (r = -0.93, 95% CI [-0.95,-0.90], p <10^-15) and a strong increasing trend of average yearly NDC (r = 0.93, 95% CI [0.91,0.95], p <10 ^-15) (Figure 2A–E). Next, we examined the relationship between the components of the readability metrics and year of publication. The average number of syllables in each word (FRE component) and the percentage of difficult words (NDC component) showed pronounced increases over years (Figure 2F,G). Sentence length (FRE and NDC component) showed a steady increase with year after 1960 (Figure 2H), the period in which the majority of abstracts were published (Figure 1B). FRE and NDC were correlated with one another (r = -0.72, 95% CI [-0.72,-0.72], p <10^-15) (Figure 2E).

Figure 2 with 1 supplement see all

Download asset Open asset

Figure 2—source data 1

Readability data of abstracts and number of authors per article.

https://doi.org/10.7554/eLife.27725.005

Download elife-27725-fig2-data1-v2.csv

Figure 2—source data 2

Readability data when no preprocessing is done.

https://doi.org/10.7554/eLife.27725.006

Download elife-27725-fig2-data2-v2.csv

The readability of individual abstracts was formally evaluated in relation to year of publication using a linear mixed effects model with journal as a random effect for both measures. The fixed effect of year was significantly related to FRE and NDC scores (Table 1). The average yearly trends combined with this statistical model reveal that the complexity of scientific writing is increasing with time. In order to explore whether this trend was consistent across all 12 selected fields, we extracted the slopes (random effects) from the mixed effects models for each journal. This showed that the trend of decreasing readability over time is not specific to any particular field, although there are differences in magnitude between fields (Figure 3). Further, only two journals out of 123 showed clear increases in FRE across time (Figure 3—figure supplement 1).

Figure 3 with 1 supplement see all

Download asset Open asset

Figure 3—source data 1

Summary of FRE and NDC journal random slopes for each field extracted from the linear mixed model (M2).

https://doi.org/10.7554/eLife.27725.010

Download elife-27725-fig3-data1-v2.csv

Figure 3—source data 2

FRE and NDC random slopes for each journal extracted from the linear mixed model (M2).

https://doi.org/10.7554/eLife.27725.011

Download elife-27725-fig3-data2-v2.csv

To verify that the readability of abstracts was representative of the readability of the entire articles, we downloaded full text articles from six additional independent journals from which all articles were available in the PubMed Central Open Access Subset (Figure 4A). Although, as has previously been reported (Dronberger and Kowitz, 1975), abstracts are less readable than the full articles, there was a strong positive relationship between readability of the abstracts and the full texts (FRE: r = 0.60, 95% CI [0.60, 0.60], p <10^-15; NDC: r = 0.63, 95% CI [0.63, 0.63], p <10^-15, Figure 4B, Figure 4—figure supplements 1 and 2). This implies that the increasing complexity of scientific writing generalizes to the full texts.

Figure 4 with 2 supplements see all

Download asset Open asset

Figure 4—source data 1

Readability data used in full text analysis.

https://doi.org/10.7554/eLife.27725.015

Download elife-27725-fig4-data1-v2.csv

There could be a number of explanations for the observed trend in scientific readability. We formulated two plausible and testable hypotheses: (1) There is an increase in the number of co-authors over time (Figure 5A) (see also (Epstein, 1993; Drenth, 1998). If the number of co-authors correlates with readability, this underlies the observed effect (i.e. a case of 'too many cooks spoil the broth'). (2) An increase in a general scientific jargon is leading to a vocabulary which is almost exclusively used by scientists and less readable in general (i.e. a 'science-ese').

Figure 5 with 1 supplement see all

Download asset Open asset

To test the first hypothesis, we divided the data by the number of authors. More authors were associated with decreased readability (Figure 5B, Figure 5—figure supplement 1A). However, we observed the same trend of decreasing readability across years regardless of the number of authors (Figure 5C, Figure 5—figure supplement 1B). When we included the number of authors as a predictor in the linear mixed effects model, it was significantly related to readability, while the fixed effect of year remained significant (Table 2). We can therefore reject the hypothesis that the increase in the number of authors on scientific articles is responsible for the observed trend, although abstract readability does decrease with more authors.

To test the second hypothesis, we constructed a measure for in-group scientific vocabulary. We selected the 2,949 most common words which were not included in the NDC common word list from 12,000 abstracts sampled at random (see Materials and methods for procedure). This is analogous to a 'science-specific common word list'. This list also includes topics which have increased over time (e.g. 'gene') and subject-specific words (e.g. 'tumor'), which are not indicative of an in-group scientific vocabulary. We removed such words to create a general scientific jargon list (2,138 words; see Materials and methods and Supplementary file 2). While the percentage of common words from the NDC common word list decreased with year (r = −0.93, 95% CI [-0.93, -0.93], p <10^-15, Figure 6A), there was an increase in the percentage of science-specific common words (r = 0.90, 95% CI [0.90, 0.91], p <10^-15) and general scientific jargon (r = 0.96, 95% CI [0.95, 0.96], p <10^-15) (Figure 6A). Twelve general science jargon words are presented in Figure 6B. While one word ('appears') decreased with time, all the remaining examples show sharp increases over time. Taken together, this provides evidence in favor of the hypothesis that there is an increase in general scientific jargon which partially accounts for the decreasing readability.

Figure 6

Download asset Open asset

Figure 6—source data 1

Frequency of words in lists and example word use per article.

https://doi.org/10.7554/eLife.27725.020

Download elife-27725-fig6-data1-v2.csv

Figure 6—source data 2

PubMed ID for files used in training and verification lists of science common word list.

https://doi.org/10.7554/eLife.27725.021

Download elife-27725-fig6-data2-v2.csv

From analyzing over 700,000 abstracts in 123 journals from the biomedical and life sciences, as well as general science journals, we have shown a steady decrease of readability over time in the scientific literature. It is important to put the magnitude of these results in context. A FRE score of 100 is designed to reflect the reading level of a 10- to 11-year old. A score between 0 and 30 is considered understandable by college graduates (Flesch, 1948; Kincaid et al., 1975). In 1960, 14% of the texts in our corpus had a FRE below 0. In 2015, this number had risen to 22%. In other words, more than a fifth of scientific abstracts now have a readability considered beyond college graduate level English. However, the absolute readability scores should be interpreted with some caution: scores can vary due to different media (e.g. comics versus news articles; Štajner et al., 2012) and education level thresholds can be imprecise (Stokes, 1978). We then validated abstract readability against full text readability, demonstrating that it is a suitable approximation for comparing main texts.

We investigated two possible reasons why this trend has occurred. First, we found that readability of abstracts correlates with the number of co-authors, but this failed to fully account for the trend through time. Second, we showed that there is an increase in general scientific jargon over years. These general science jargon words should be interpreted as words which scientists frequently use in scientific texts, and not as subject specific jargon. This finding is indicative of a progressively increasing in-group scientific language ('science-ese').

An alternative explanation for the main finding is that the cumulative growth of scientific knowledge makes an increasingly complex language necessary. This cannot be directly tested, but if this were to fully explain the trend, we would expect a greater diversity of vocabulary as science grows more specialized. While accounting for the original finding of the increase in difficult words and of syllable count, this would not explain the increase of general scientific jargon words (e.g. 'furthermore' or 'novel', Figure 6B). Thus, this possible explanation cannot fully account for our findings.

Lower readability implies less accessibility, particularly for non-specialists, such as journalists, policy-makers and the wider public. Scientific journalism offers a key role in communicating science to the wider public (Bubela et al., 2009) and scientific credibility can sometimes suffer when reported by journalists (Hinnant and Len-Ríos, 2009). Considering this, decreasing readability cannot be a positive development for efforts to accurately communicate science to non-specialists. Further, amidst concerns that modern societies are becoming less stringent with actual truths, replaced with true-sounding 'post-facts' (Manjoo, 2011; Nordenstedt and Rosling, 2016) science should be advancing our most accurate knowledge. One way to achieve this is for science to maximize its accessibility to non-specialists.

Lower readability is also a problem for specialists (Hartley, 1994; Hartley and Benjamin, 1998; Hartley, 2003). This was explicitly shown by Hartley (1994) who demonstrated that rewriting scientific abstracts, to improve their readability, increased academics’ ability to comprehend them. While science is complex, and some jargon is unavoidable (Knight, 2003), this does not justify the continuing trend that we have shown. It is also worth considering the importance of comprehensibility of scientific texts in light of the recent controversy regarding the reproducibility of science (Prinz et al., 2011; McNutt, 2014; Begley and Ioannidis, 2015; Nosek et al., 2015; Camerer et al., 2016). Reproducibility requires that findings can be verified independently. To achieve this, reporting of methods and results must be sufficiently understandable.

Readability formulas are not without their limitations. They provide an estimate of a text’s readability and should not be interpreted as a categorical measure of how well a text will be understood. For example, readability can be affected by text size, line spacing, the use of headers, as well as by the use of visual aids such as tables or graphs, none of which are captured by readability formulas (Hartley, 2013; Badarudeen and Sabharwal, 2010). Many semantic properties of texts are overlooked, including the complexity of ideas, the rhetorical structure and the overall coherence of the text (Bruce et al., 1981; Danielson, 1987; Zamanian and Heydari, 2012). Changing a text solely to improve readability scores does not automatically make a text more understandable (Duffy and Kabance, 1982; Redish, 2000).

Despite the limitations of readability formulas, our study shows that recent scientific texts are, on average, less readable than older scientific texts. This trend was not specific to any one field, even though the size of this association varied across fields. Some fields also had a steeper decline in NDC scores, while less of a decline in FRE scores, and vice versa (Figure 3). Further research should explore possible reasons for these differences, as it may give clues on how to improve readability. For example, the adoption of structured abstracts which are known to assist readability (Hartley and Benjamin, 1998; Hartley, 2003) might lead to a less steep decline for some fields.

What more can be done to reverse this trend? The emerging field of science communication deals with ways science can effectively communicate ideas to a wider audience (Treise and Weigold, 2002; Nielsen, 2013; Fischhoff, 2013). One suggestion from this field is to create accessible 'lay summaries', which have been implemented by some journals (Kuehne and Olden, 2015). Others have noted that scientists are increasing their direct communication with the general public through social media (Peters et al., 2014), and this trend could be encouraged. However, while these two suggestions may increase accessibility of scientific results, neither will reverse the readability trend of scientific texts. Another proposal is to make scientific communication a necessary part of undergraduate and graduate education (Brownell et al., 2013).

Scientists themselves can estimate their own readability in most word processing software. Further, while some journals aim for high readability, perhaps a more thorough review of article readability should be carried out by journals in the review process. Finally, in an era of data metrics, it is possible to assess a scientist’s average readability, analogous to the h-index for citations (Hirsch, 2005). Such an 'r-index' could be considered an asset for those scientists who emphasize clarity in their writing.

1. 1. PubMed

   (2016) PMC Open Access Subset

   Publicly available at the NCBI ftp site for oa_bulk.

   ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk/

1. 1. Badarudeen S
   2. Sabharwal S

   (2010) Assessing readability of patient education materials: current role in orthopaedics

   Clinical Orthopaedics and Related Research 468:2572–2580.

   https://doi.org/10.1007/s11999-010-1380-y

   • PubMed
   • Google Scholar
2. 1. Bates D
   2. Mächler M
   3. Bolker B
   4. Walker S

   (2014)

   Fitting Linear Mixed-Effects Models using lme4

   Journal of Statistical Software 67:51.

   • Google Scholar
3. 1. Begley CG
   2. Ioannidis JP

   (2015) Reproducibility in science: improving the standard for basic and preclinical research

   Circulation Research 116:116–126.

   https://doi.org/10.1161/CIRCRESAHA.114.303819

   • PubMed
   • Google Scholar
4. 1. Benjamin RG

   (2012) Reconstructing readability: recent developments and recommendations in the analysis of text difficulty

   Educational Psychology Review 24:63–88.

   https://doi.org/10.1007/s10648-011-9181-8

   • Google Scholar
5. Book

   1. Bird S
   2. Klein E
   3. Lower E
   4. Loper E

   (2009)

   Natural Language Processing with Python, 43

   O’Reilly Media.

   • Google Scholar
6. 1. Brownell SE
   2. Price JV
   3. Steinman L

   (2013)

   Science communication to the general public: Why we need to teach undergraduate and graduate students this skill as part of their formal scientific training

   Journal of Undergraduate Neuroscience Education 12:E6–E10.

   • PubMed
   • Google Scholar
7. 1. Bruce B
   2. Rubin A
   3. Starr K

   (1981) Why readability formulas fail

   IEEE Transactions on Professional Communication 24:50–52.

   https://doi.org/10.1109/TPC.1981.6447826

   • Google Scholar
8. 1. Bubela T
   2. Nisbet MC
   3. Borchelt R
   4. Brunger F
   5. Critchley C
   6. Einsiedel E
   7. Geller G
   8. Gupta A
   9. Hampel J
   10. Hyde-Lay R
   11. Jandciu EW
   12. Jones SA
   13. Kolopack P
   14. Lane S
   15. Lougheed T
   16. Nerlich B
   17. Ogbogu U
   18. O'Riordan K
   19. Ouellette C
   20. Spear M
   21. Strauss S
   22. Thavaratnam T
   23. Willemse L
   24. Caulfield T

   (2009) Science communication reconsidered

   Nature Biotechnology 27:514–518.

   https://doi.org/10.1038/nbt0609-514

   • PubMed
   • Google Scholar
9. 1. Camerer CF
   2. Dreber A
   3. Forsell E
   4. Ho TH
   5. Huber J
   6. Johannesson M
   7. Kirchler M
   8. Almenberg J
   9. Altmejd A
   10. Chan T
   11. Heikensten E
   12. Holzmeister F
   13. Imai T
   14. Isaksson S
   15. Nave G
   16. Pfeiffer T
   17. Razen M
   18. Wu H

   (2016) Evaluating replicability of laboratory experiments in economics

   Science 351:1433–1436.

   https://doi.org/10.1126/science.aaf0918

   • PubMed
   • Google Scholar
10. Book

    1. Chall JS
    2. Dale E

    (1995)

    Readability revisited: The New Dale-Chall readability formula

    Brookline Books.

    • Google Scholar
11. 1. Danielson KE

    (1987)

    Readability formulas: a necessary evil?

    Reading Horizons 27:.

    • Google Scholar
12. 1. Danielson WA
    2. Lasorsa DL
    3. Im DS

    (1992) Journalists and Novelists: A Study of Diverging Styles

    Journalism Quarterly 69:436–446.

    https://doi.org/10.1177/107769909206900217

    • Google Scholar
13. 1. Didegah F
    2. Thelwall M

    (2013) Which factors help authors produce the highest impact research? Collaboration, journal and document properties

    Journal of Informetrics 7:861–873.

    https://doi.org/10.1016/j.joi.2013.08.006

    • Google Scholar
14. 1. Drenth JPH

    (1998) Multiple Authorship: The Contribution of Senior Authors

    JAMA 280:219.

    https://doi.org/10.1001/jama.280.3.219

    • Google Scholar
15. 1. Dronberger GB
    2. Kowitz GT

    (1975) Abstract readability as a factor in information systems

    Journal of the American Society for Information Science 26:108–111.

    https://doi.org/10.1002/asi.4630260206

    • Google Scholar
16. Book

    1. DuBay WH

    (2004)

    The principles of readability

    Impact Information.

    • Google Scholar
17. 1. Duffy TM
    2. Kabance P

    (1982) Testing a readable writing approach to text revision

    Journal of Educational Psychology 74:733–748.

    https://doi.org/10.1037/0022-0663.74.5.733

    • Google Scholar
18. 1. Epstein RJ

    (1993) Six authors in search of a citation: villains or victims of the Vancouver convention?

    BMJ 306:765–767.

    https://doi.org/10.1136/bmj.306.6880.765

    • Google Scholar
19. 1. Fischhoff B

    (2013) The sciences of science communication

    PNAS 110 Suppl 3:14033–14039.

    https://doi.org/10.1073/pnas.1213273110

    • PubMed
    • Google Scholar
20. 1. Flesch R

    (1948) A new readability yardstick

    Journal of Applied Psychology 32:221–233.

    https://doi.org/10.1037/h0057532

    • PubMed
    • Google Scholar
21. 1. Hartley J

    (1994) Three ways to improve the clarity of journal abstracts

    British Journal of Educational Psychology 64:331–343.

    https://doi.org/10.1111/j.2044-8279.1994.tb01106.x

    • Google Scholar
22. 1. Hartley J
    2. Benjamin M

    (1998) An evaluation of structured abstracts in journals published by the British Psychological Society

    British Journal of Educational Psychology 68:443–456.

    https://doi.org/10.1111/j.2044-8279.1998.tb01303.x

    • Google Scholar
23. 1. Hartley J

    (2003) Improving the clarity of journal abstracts in psychology: the case for structure

    Science Communication 24:366–379.

    https://doi.org/10.1177/1075547002250301

    • Google Scholar
24. Book

    1. Hartley J

    (2013)

    Designing Instructional Text

    RoutledgeFalmer.

    • Google Scholar
25. 1. Hayes DP

    (1992) The growing inaccessibility of science

    Nature 356:739–740.

    https://doi.org/10.1038/356739a0

    • Google Scholar
26. 1. Hinnant A
    2. Len-Ríos ME

    (2009) Tacit understandings of health literacy

    Science Communication 31:84–115.

    https://doi.org/10.1177/1075547009335345

    • Google Scholar
27. 1. Hirsch JE

    (2005) An index to quantify an individual's scientific research output

    PNAS 102:16569–16572.

    https://doi.org/10.1073/pnas.0507655102

    • PubMed
    • Google Scholar
28. 1. Jatowt A
    2. Tanaka K

    (2012)

    Longitudinal analysis of historical texts’ readability

    Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries pp. 353–354.

    • Google Scholar
29. 1. Kincaid JP
    2. Fishburne RP
    3. Rogers RL
    4. Chissom BS

    (1975)

    Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel

    Naval Technical Training Command Millington TN Research Branch. RBR-8-75.

    • Google Scholar
30. 1. Knight J

    (2003) Scientific literacy: Clear as mud

    Nature 423:376–378.

    https://doi.org/10.1038/423376a

    • PubMed
    • Google Scholar
31. Software

    1. Kovalchik S

    (2016)

    RISmed: Download Content from NCBI Databases, version 2.1.6

    R Package Version.
32. 1. Kuehne LM
    2. Olden JD

    (2015) Opinion: Lay summaries needed to enhance science communication

    PNAS 112:3585–3586.

    https://doi.org/10.1073/pnas.1500882112

    • PubMed
    • Google Scholar
33. 1. Kuznetsova A
    2. Brockhoff PB
    3. Christensen RHB

    (2015)

    Package 'lmerTest’

    Package 'lmerTest’, 17.

    • Google Scholar
34. Book

    1. Lim ET

    (2008) The Anti-Intellectual Presidency: The Decline of Presidential Rhetoric From George Washington To George W Bush

    Oxford University Press.

    https://doi.org/10.1093/acprof:oso/9780195342642.001.0001

    • Google Scholar
35. Book

    1. Manjoo F

    (2011)

    True Enough: Learning to Live in a Post-Fact Society

    John Wiley & Sons.

    • Google Scholar
36. 1. McNutt M

    (2014) Reproducibility

    Science 343:229.

    https://doi.org/10.1126/science.1250475

    • PubMed
    • Google Scholar
37. 1. Nielsen KH

    (2013) Scientific communication and the nature of science

    Science & Education 22:2067–2086.

    https://doi.org/10.1007/s11191-012-9475-3

    • Google Scholar
38. 1. Nordenstedt H
    2. Rosling H

    (2016) Chasing 60{%} of maternal deaths in the post-fact era

    The Lancet 388:1864–1865.

    https://doi.org/10.1016/S0140-6736(16)31793-7

    • Google Scholar
39. 1. Nosek BA
    2. Alter G
    3. Banks GC
    4. Borsboom D
    5. Bowman SD
    6. Breckler SJ
    7. Buck S
    8. Chambers CD
    9. Chin G
    10. Christensen G
    11. Contestabile M
    12. Dafoe A
    13. Eich E
    14. Freese J
    15. Glennerster R
    16. Goroff D
    17. Green DP
    18. Hesse B
    19. Humphreys M
    20. Ishiyama J
    21. Karlan D
    22. Kraut A
    23. Lupia A
    24. Mabry P
    25. Madon T
    26. Malhotra N
    27. Mayo-Wilson E
    28. McNutt M
    29. Miguel E
    30. Paluck EL
    31. Simonsohn U
    32. Soderberg C
    33. Spellman BA
    34. Turitto J
    35. VandenBos G
    36. Vazire S
    37. Wagenmakers EJ
    38. Wilson R
    39. Yarkoni T

    (2015) Promoting an open research culture

    Science 348:1422–1425.

    https://doi.org/10.1126/science.aab2374

    • Google Scholar
40. 1. Peters HP
    2. Dunwoody S
    3. Allgaier J
    4. Lo YY
    5. Brossard D

    (2014) Public communication of science 2.0: Is the communication of science via the "new media" online a genuine transformation or old wine in new bottles?

    EMBO Reports 15:749–753.

    https://doi.org/10.15252/embr.201438979

    • PubMed
    • Google Scholar
41. Software

    1. Plavén-Sigray P
    2. Matheson GJ
    3. Schiffler BC
    4. Thompson WH

    (2017) readabilityinscience, version 739fd2a

    Github.

    https://www.github.com/wiheto/readabilityinscience
42. 1. Prinz F
    2. Schlange T
    3. Asadullah K

    (2011) Believe it or not: how much can we rely on published data on potential drug targets?

    Nature Reviews Drug Discovery 10:712.

    https://doi.org/10.1038/nrd3439-c1

    • PubMed
    • Google Scholar
43. 1. Redish JC
    2. Selzer J

    (1985)

    The place of readability formulas in technical communication

    Journal of Reading Behavior 32:46–52.

    • Google Scholar
44. 1. Redish J

    (2000) Readability formulas have even more limitations than Klare discusses

    ACM Journal of Computer Documentation 24:132–137.

    https://doi.org/10.1145/344599.344637

    • Google Scholar
45. 1. Schmid H

    (1994)

    Proceedings of the International Conference on New Methods in Language Processing

    44–49, Probabilistic part-of-speech tagging using decision trees, Proceedings of the International Conference on New Methods in Language Processing.

    • Google Scholar
46. 1. Štajner S
    2. Evans R
    3. Orăsan C
    4. Mitkov R

    (2012)

    Workshop on Natural Language Processing for Improving Textual Accessibility

    14–21, What can readability measures really tell us about text complexity?, Workshop on Natural Language Processing for Improving Textual Accessibility.

    • Google Scholar
47. 1. Stevenson RL

    (1964) Readability of conservative and sensational papers since 1872

    Journalism Quarterly 41:201–206.

    https://doi.org/10.1177/107769906404100206

    • Google Scholar
48. 1. Stokes A

    (1978) The reliability of readability formulae

    Journal of Research in Reading 1:21–34.

    https://doi.org/10.1111/j.1467-9817.1978.tb00170.x

    • Google Scholar
49. 1. Treise D
    2. Weigold MF

    (2002) Advancing science communication

    Science Communication 23:310–322.

    https://doi.org/10.1177/107554700202300306

    • Google Scholar
50. 1. Vinkers CH
    2. Tijdink JK
    3. Otte WM

    (2015) Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis

    BMJ 351:h6467.

    https://doi.org/10.1136/bmj.h6467

    • PubMed
    • Google Scholar
51. 1. Zamanian M
    2. Heydari P

    (2012) Readability of texts: state of the art

    Theory and Practice in Language Studies 2:43–53.

    https://doi.org/10.4304/tpls.2.1.43-53

    • Google Scholar

Thank you for submitting your article "The readability of scientific texts is decreasing over time" for consideration as a Feature Article by eLife. Your article has been reviewed by two peer reviewers, Ralf Barkemeyer (Reviewer #1) and Christiaan Vinkers (Reviewer #2), and an Associate Features Editor, Stuart King.

The reviewers have discussed the reviews with one another and the Associate Features Editor has drafted this decision to help you prepare a revised submission. We hope you will be able to submit the revised version within two months.

Summary

This manuscript – which first appeared as a preprint on bioRxiv – reports a decline in the readability of scientific texts meaning that, on average, papers published in recent decades are harder to read than those from over a century ago. Two possible explanations are tested: an increase in the number of authors per manuscript, and the wider use of uncommon language (referred to as "general scientific jargon"). All three reviewers agree that the manuscript is timely and relevant, and that it has the potential to make a clear contribution to the existing literature. The research design looks sound, and mainly builds on two well-established measures of readability. The analyses appear to be executed in a rigorous and transparent manner, and the authors generate a set of straightforward and intuitive but highly relevant findings. The reviewers hope that this manuscript will raise awareness, and encourage scientists and publishers to take readability into account when authoring or reviewing scientific papers.

Nevertheless, there are some issues that need to be addressed, and several revisions that would strengthen the manuscript. Most can be handled by editing the text to provide more details or clarity, or by improving the presentation of the figures. One will require a more fine-grained analysis of the data that is already available.

Essential Revisions:

1. Both of the readability formulas reported in the Materials and methods section contain mistakes. The constant in the Flesch Reading Ease (FRE) formula is incorrectly given as "206.385" instead of "206.835". The New Dale-Chall (NDC) Readability Formula is repeatedly missing the "× 100" that converts the fraction of "difficult words" to a percentage. The inequality symbols (i.e. > and {less than or equal to}) have also been transposed in the NDC formula meaning that, if followed as written, the scores would not be adjusted properly. From the data provided, it is clear that the correct NDC formula was used for the analyses. But we cannot check the FRE calculations because the "syllable counts" are missing from the data provided (see point 4). Please confirm that the correct FRE formula was used and fix the mistakes described above.

2. The readability measures chosen appear plausible and test two complementary dimensions of readability. However, please provide some more justification as to why these two measures were chosen and not others.

3. Please elaborate on the limitations of readability metrics in general, and those linked to text pre-processing. For example, visual aids are not captured even though they might improve comprehension; or the fact that FRE does not take into account parentheses and other ways of structuring text (which might also aid readability).

4. The authors have agreed to make the data and scripts openly accessible, which is applaudable. However, the data provided with the full submission is missing the syllable counts. Please provide all the data needed to recreate the figures in this version of the manuscript. This also includes the analyses related to the full texts, number of authors, and "general scientific jargon". Please see the section on "Source Data Files" in eLife's author guide for more details.

5. The current Figures 1 and 3are packed with data and somewhat difficult to follow. Please divide them each into two separate figures. For Figure 1, separate the summary graphics from the panels showing the main trends. For Figure 3, the data related to number of authors should be separated from that related to "general scientific jargon".

6. The phrase "high-impact journals" is used without explanation. The specific meaning of the word "impact" in this context is unclear and could arguably be considered jargon. Please explain this phrase in the text or replace with a more accurate description like "highly cited journals".

7. Please elaborate on the criteria used to select the "six representative" open-access journals for the full-text analysis. Please also provide information equivalent to that provided for the journals in the abstract-only analysis.

8. As it stands now, the analysis remains at a very high level. Digging deeper into the existing data might reveal avenues to pursue to address the poor readability of scientific texts. Please report and discuss the trends in readability (or the average values) for the different journals and/or fields of research. For example, showing that articles in some journals are more readable than others might encourage someone to trace these differences back to specific editorial policies (word limits etc.) that could then be adopted more widely.

9. The literature on science communication is extensive, covering both the underlying communication process as well as its implications. Please strengthen the manuscript's links to this literature; for example, by discussing to what extent is it necessary that non-specialists can read and understand scientific articles, given that mainstream media and other new outlets have been created to specifically focus on non-specialist audiences in this context. While the reviewers likely agree with the authors' line of argument, some more contextualization and justification would be helpful here.

10. Please integrate the "Supplementary methods" into the "Materials and methods" section, to avoid the need to jump back and forth in the text. The sections on selecting journal fields and identifying "candidate jargon" would also benefit from being critically edited to improve clarity.

11. On a related point, please define classifications like "science-specific common words" and "candidate general science jargon" more clearly, and then consistently use them throughout the text. As currently written, some readers would not recognize uncommon/formal words - such as "novel", "robust", "moreover" and "therefore" - as scientific jargon. It would also be helpful if the discussion on this topic were expanded.

12. The FRE score of 48.2 for the submitted manuscript is respectable. However, please look for further opportunities to make the article easier to read, including ways that would not necessarily get picked up by the readability formulas referenced in the article. For example, while most English speakers are likely to know the verb "understand", fewer will be familiar with the adjective "understandable", and fewer still the noun "understandability" (which is used in the manuscript's abstract). Please consider using fewer nouns and more verbs throughout the abstract, main text and figure captions (e.g. "Scientific abstracts have become harder to read over time" rather than "Readability of scientific abstracts decreases over time".)

Essential Revisions:

1. Both of the readability formulas reported in the Materials and methods section contain mistakes. The constant in the Flesch Reading Ease (FRE) formula is incorrectly given as "206.385" instead of "206.835". The New Dale-Chall (NDC) Readability Formula is repeatedly missing the "× 100" that converts the fraction of "difficult words" to a percentage. The inequality symbols (i.e. > and {less than or equal to}) have also been transposed in the NDC formula meaning that, if followed as written, the scores would not be adjusted properly. From the data provided, it is clear that the correct NDC formula was used for the analyses. But we cannot check the FRE calculations because the "syllable counts" are missing from the data provided (see point 4). Please confirm that the correct FRE formula was used and fix the mistakes described above.

We thank the reviewers for noting the typos in the readability formulas.

These mistakes were not present in the analysis code and have now been corrected in the text. We further apologise for the syllable count being omitted from the source data we included. This has been fixed and included as source data (see the response to point 4 for a further details regarding the source data).

2. The readability measures chosen appear plausible and test two complementary dimensions of readability. However, please provide some more justification as to why these two measures were chosen and not others.

Our reason for using two measures was to make sure the result was not merely induced by the measure itself. The reasons for using FRE were because of its popularity and because it has been used in looking at temporal trends in other studies. The reason for using NDC was because it includes something different to FRE in how it is calculated (the common word lists). Further it performs comparatively compared to recently proposed readability measures. We have added some sentences to justify the choice of two readability metrics in the first paragraph of the “Language and Readability metrics” subsection of the Material and methods section.

3. Please elaborate on the limitations of readability metrics in general, and those linked to text pre-processing. For example, visual aids are not captured even though they might improve comprehension; or the fact that FRE does not take into account parentheses and other ways of structuring text (which might also aid readability).

We agree that it is important to emphasize the limitations of readability formulas. We have therefore amended the limitations paragraph with examples for text elements which contribute to understanding but which are often removed during pre-processing or not captured by readability formulas.

4. The authors have agreed to make the data and scripts openly accessible, which is applaudable. However, the data provided with the full submission is missing the syllable counts. Please provide all the data needed to recreate the figures in this version of the manuscript. This also includes the analyses related to the full texts, number of authors, and "general scientific jargon". Please see the section on "Source Data Files" in eLife's author guide for more details.

Source data is now provided for Figures 2, 3, 4 and 6.

We have now added the summed syllable-count to the “Figure 2—source data 1”, which provides all the information for FRE and NDC to be calculated per article.

We have further added “number of authors” to the same file. We have added a similar file: “Figure 2—source data 2” for the analysis on the unpreprocessed data. This allows for full recreation of the relevant figures (Figure 2 and its supplement).

“Figure 3—source data 1” and “Figure 3—source data 2” is now included and will allow reproduction of Figure 3 and its supplement.

“Figure 4—source data 1” is now included and allows reproduction of Figure 4 and its supplements.

“Figure 6—source data 1” contains all the information needed to reproduce Figure 6.

“Figure 6—source data 2” is also included which allows for the analysis to be reproduced regarding the derivation of the science easy words.

Figure 5 and Figure 1 both use “Figure 2—source data 1” as their source data.

It is associated with Figure 2 as most of the data in the file belongs to Figure 2. In the figure captions of both Figure 1 and Figure 5 it is noted that “Figure 2—source data 1” is where the source data can be found.

In sum all figures with data now have relevant source data that can be used to recreate the figures. Finally, we have placed all the scripts on a public repository on GitHub: https://github.com/wiheto/readabilityinscience and these are now available.

5. The current Figures 1 and 3 are packed with data and somewhat difficult to follow. Please divide them each into two separate figures. For Figure 1, separate the summary graphics from the panels showing the main trends. For Figure 3, the data related to number of authors should be separated from that related to "general scientific jargon".

We have divided up the figures as requested. There are now six figures in total: the summary graphic (Figure 1); the main readability result (Figure 2); the random effects per field (Figure 3; and per journal in the figure supplement); the full text data (Figure 4); the number of authors (Figure 5); the “jargon” analysis (Figure 6).

6. The phrase "high-impact journals" is used without explanation. The specific meaning of the word "impact" in this context is unclear and could arguably be considered jargon. Please explain this phrase in the text or replace with a more accurate description like "highly cited journals".

The reviewers are correct that this term is unclear, and this term is certainly a form of science-ese. The text has been amended to say “highly cited” both in the introduction and methods.

7. Please elaborate on the criteria used to select the "six representative" open-access journals for the full-text analysis. Please also provide information equivalent to that provided for the journals in the abstract-only analysis.

We have expanded upon the reasons for why these particular six journals have been included in the abstract-full text analysis. Importantly, one of the main reasons to choose these journals was because they were Open Access - and Open Access journals are the only ones that allow us to download full texts en masse without violating the terms and conditions of services like Pubmed. We have added a supplementary file to the manuscript (Supplementary file 4), which displays information about the journals included in this analysis. Further, we have also now provided additional supplementary figures (Figure 4—figure supplement 2), demonstrating that the observed correlation between readability for abstracts and full texts is present in each of the journals included in the analysis.

8. As it stands now, the analysis remains at a very high level. Digging deeper into the existing data might reveal avenues to pursue to address the poor readability of scientific texts. Please report and discuss the trends in readability (or the average values) for the different journals and/or fields of research. For example, showing that articles in some journals are more readable than others might encourage someone to trace these differences back to specific editorial policies (word limits etc.) that could then be adopted more widely.

We agree with the reviewers that a deeper analysis can yield ideas and hypotheses for future research on how to improve readability. We have therefore extracted the random effect (i.e. slope) of each journal from the mixed models and plotted them both by themselves, and as field-summaries, in Figure 3 and its supplement. This shows that the trend is present in all fields, although some journals do not show a decline in readability. We have amended the results section accordingly, and added a discussion about the possibility of future research to examine why there are differences between fields and journals, and how this could yield clues on how to address the problem.

9. The literature on science communication is extensive, covering both the underlying communication process as well as its implications. Please strengthen the manuscript's links to this literature; for example, by discussing to what extent is it necessary that non-specialists can read and understand scientific articles, given that mainstream media and other new outlets have been created to specifically focus on non-specialist audiences in this context. While the reviewers likely agree with the authors' line of argument, some more contextualization and justification would be helpful here.

In answer to this point we have rewritten two paragraphs. The paragraph that begins “Lower readability implies less accessibility, particularly for non-specialists, such as journalists, policy-makers and the wider public” (Discussion, fourth paragraph) now clarifies that “non-specialists” also applies to journalists within the mainstream media. This should clarify that this even affects those that use science journalism as a news source. We have also changed the focus of this paragraph from previously talking about scientific literacy and now focus more on scientific journalism. This is because it is hard to see that scientific literacy is causally affected by our results here, and it was a speculative link of troubling trends.

Additionally the paragraph that begins “What more can be done to reverse this trend?” (Discussion, penultimate paragraph) makes greater links with the literature in scientific communication. We state some of the more prominent proposals that have been made to make scientific texts more accessible.

10. Please integrate the "Supplementary methods" into the "Materials and methods" section, to avoid the need to jump back and forth in the text. The sections on selecting journal fields and identifying "candidate jargon" would also benefit from being critically edited to improve clarity.

The two sections that were previously in the Supplementary Methods, namely the selection of journal fields and identifying candidate jargon have both been moved to the Materials and methods section. Both sections have been made shorter and clearer.

11. On a related point, please define classifications like "science-specific common words" and "candidate general science jargon" more clearly, and then consistently use them throughout the text. As currently written, some readers would not recognize uncommon/formal words - such as "novel", "robust", "moreover" and "therefore" - as scientific jargon. It would also be helpful if the discussion on this topic were expanded.

The Material and methods subsection “Science-specific common word and general science jargon lists” has been completely rewritten where we give clear definitions of what these two terms mean. We have also dropped the term “candidate” from the entire text as it could be considered confusing. The initial reason for including it was to not definitively declare these terms as jargon, since they have been defined only by excluding certain categories of words. An additional sentence has been added to the discussion paragraph to clarify how to interpret general scientific jargon.

12. The FRE score of 48.2 for the submitted manuscript is respectable. However, please look for further opportunities to make the article easier to read, including ways that would not necessarily get picked up by the readability formulas referenced in the article. For example, while most English speakers are likely to know the verb "understand", fewer will be familiar with the adjective "understandable", and fewer still the noun "understandability" (which is used in the manuscript's abstract). Please consider using fewer nouns and more verbs throughout the abstract, main text and figure captions (e.g. "Scientific abstracts have become harder to read over time" rather than "Readability of scientific abstracts decreases over time".)

We have now gone through the manuscript and, to the best of our ability, tried to improve the readability further. For example, sentences have been shortened, nouns have been replaced by verbs when suitable, and complicated words have been replaced by simpler synonyms when this could be done without losing meaning. Further, some parts of the text have been completely rewritten to make them easier to understand, such as the entire journal selection procedure paragraph in the Material and methods section. Please see the track-changed document for all readability improvements in the manuscript.

In addition to the revisions detailed above requested by the reviewers, we identified three additional issues with the article:

There was a typo in the FRE calculation of a scalar in the analysis code. The part “1.015 * words/sentences” was incorrectly set to 1.1015. This has led to a very slight increase in all FRE scores when fixed. This change has had no effect on the observed trend. However, the sentence “In other words, more than a quarter of scientific abstracts now have a readability considered beyond college graduate level English.” (Discussion, first paragraph) has been changed to “In other words, more than a fifth of scientific abstracts now have a readability considered beyond college graduate level English.”

The selection criteria state that 12 journals from each field were included. However, for the field Immunology, only 11 journals had been included by mistake. The Journal “Brain Behav Immun” has now been added. This has increased the number of abstracts from 707,452 to 709,577 and all values reported in the article have been updated accordingly. This includes a new training and verification set for the “general science jargon” analysis, for which there are now 2,138 general jargon words instead of 2,140 previous ones. These changes have not led to any difference in the interpretation of the results.

We noticed that the language parameters for the TreeTagger software used for preprocessing differed slightly between Linux and Windows versions. Previously the full text analysis had been run on Windows and the abstract analysis on Linux. The full text analysis has now also been run on Linux. This has led to minor differences in readability scores for the full text analysis. These minor differences have no impact on the interpretation of the results. We have also noted relevant package version numbers and the operating system used in the Material and methods section.

In sum these three changes have no impact on the conclusions drawn from the overall analysis. They however ensure that the analysis is more correct and easier to reproduce.

Author details

1. Pontus Plavén-Sigray

   Pontus Plavén-Sigray is in the Department of Clinical Neuroscience, Karolinska Institutet, Stockholm,Sweden

   Contribution

   Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing, Contributed to all aspects of the work. Contributed equally and their author order was determined randomly

   Contributed equally with

   Granville James Matheson and Björn Christian Schiffler

   Competing interests

   No competing interests declared

2. Granville James Matheson

   Granville James Matheson is in the Department of Clinical Neuroscience, Karolinska Institutet, Stockholm,Sweden

   Contribution

   Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing, Contributed to all aspects of the work. Contributed equally and their author order was determined randomly

   Contributed equally with

   Pontus Plavén-Sigray and Björn Christian Schiffler

   Competing interests

   No competing interests declared

3. Björn Christian Schiffler

   Björn Christian Schiffler is in the Department of Clinical Neuroscience, Karolinska Institutet, Stockholm,Sweden

   Contribution

   Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing, Contributed to all aspects of the work. Contributed equally and their author order was determined randomly

   Contributed equally with

   Pontus Plavén-Sigray and Granville James Matheson

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2347-4965

4. William Hedley Thompson

   William Hedley Thompson is in the Department of Clinical Neuroscience, Karolinska Institutet, Stockholm,Sweden

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing, Conceived the study and wrote the majority of the analysis pipeline. Contributed to all aspects of the work

   For correspondence

   hedley@startmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0533-6035

• 37,553

  views

• 2,285

  downloads

• 154

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.