Neuroscience

Science Forum: How failure to falsify in high-volume science contributes to the replication crisis

College of Information Sciences and Technology, The Pennsylvania State University, United States
Center for Open Science, United States
Department of Psychology and the Social Life and Engineering Sciences Imaging Center, The Pennsylvania State University, United States

Aug 8, 2022

https://doi.org/10.7554/eLife.78830

Open access
Copyright information

Download
Cite
CommentOpen annotations (there are currently 0 annotations on this page).
Share

2 figures and 1 table

Figures

Figure 1

Download asset Open asset

Two competing theories for functional network response after brain injury.

Panel A represents the typical pattern of resting connectivity for the default mode network (DMN) and the yellow box shows a magnified area of neuronal bodies and their axonal projections. Panel B reveals three active neuronal projections (red) that are then disrupted by hemorrhagic lesion of white matter (Panel C). In response to this injury, a hyperconnectivity response (Panel D, left) shows increased signaling to adjacent areas resulting in a pronounced DMN response (Panel D, right). By contrast a disconnection hypothesis maintains that signaling from the original neuronal assemblies is diminished due to axonal degradation and neuronal atrophy secondary to cerebral diaschisis (Panel E, left) resulting in reduced functional DMN response (Panel E, right).

Figure 2

Download asset Open asset

The role of falsification in pruning high volume science to identify the fittest theories.

Panels A and B illustrate the conceptual steps in theory progression from exploration through confirmation and finally application. The x-axis is theoretical progression (time) and the y-axis is the number of active theories. Panel A depicts progression in the absence of falsification with continued branching of theories in the absence of pruning (theory reduction through falsification). By contrast the “Confirmatory Stage” in Panel B includes direct testing and refutation of theories/explanations resulting in only the fittest theories to choose from during application. Note: both Panels A and B include replication, but falsification during the “confirmation” phase results in a linear pathway and fewer choices from the “fittest” theories at the applied stage.

Tables

Table 1

Examples of hypotheses of different strength.

Exploratory research does not generally involve testing a hypothesis. A Testable Association is a weak hypothesis as it is difficult to refute. A Testable/Falsifiable Position is stronger, and a hypothesis that is Testable/Falsifiable with Alternative Finding is stronger still.

Type of research/hypothesis	Example
Exploratory	“We examine the neural correlates of cognitive deficit after brain injury implementing graph theoretical measures of whole brain neural networks”
Testable Association	“We hypothesize that graph theoretical measures of whole brain neural networks predict cognitive deficit after brain injury”
Testable/Falsifiable Position (offers possible mechanism and direction/magnitude of expected finding)	“We hypothesize that memory deficits during the first 6 months post injury are due to white matter connection loss and maintain a linear and positive relationship with increased global network path length”
Testable/Falsifiable with Alternative Finding (indicates how the hypothesis would and would not be supported)	“We hypothesize that memory deficits during the first 6 months post injury are due to white matter connection loss and maintain a linear and positive relationship with increased global network path length. Diminished global path length in individuals with greatest memory impairment would challenge this hypothesis”