In science, it is of vital importance that multiple studies corroborate the same result. Researchers therefore need to know all the details of previous experiments in order to implement the procedures as exactly as possible. However, this is becoming a major problem in neuroscience, as animal studies of behavior have proven to be hard to reproduce, and most experiments are never replicated by other laboratories.

Mice are increasingly being used to study the neural mechanisms of decision making, taking advantage of the genetic, imaging and physiological tools that are available for mouse brains. Yet, the lack of standardized behavioral assays is leading to inconsistent results between laboratories. This makes it challenging to carry out large-scale collaborations which have led to massive breakthroughs in other fields such as physics and genetics.

To help make these studies more reproducible, the International Brain Laboratory (a collaborative research group) et al. developed a standardized approach for investigating decision making in mice that incorporates every step of the process; from the training protocol to the software used to analyze the data. In the experiment, mice were shown images with different contrast and had to indicate, using a steering wheel, whether it appeared on their right or left. The mice then received a drop of sugar water for every correction decision. When the image contrast was high, mice could rely on their vision. However, when the image contrast was very low or zero, they needed to consider the information of previous trials and choose the side that had recently appeared more frequently.

This method was used to train 140 mice in seven laboratories from three different countries. The results showed that learning speed was different across mice and laboratories, but once training was complete the mice behaved consistently, relying on visual stimuli or experiences to guide their choices in a similar way.

These results show that complex behaviors in mice can be reproduced across multiple laboratories, providing an unprecedented dataset and open-access tools for studying decision making. This work could serve as a foundation for other groups, paving the way to a more collaborative approach in the field of neuroscience that could help to tackle complex research challenges.

Progress in science depends on reproducibility and thus requires standardized assays whose methods and results can be readily shared, compared, and reproduced across laboratories (Baker, 2016; Ioannidis, 2005). Such assays are common in fields such as astronomy (Fish et al., 2016; Abdalla et al., 2018), physics (CERN Education, Communications and Outreach Group, 2018), genetics (Dickinson et al., 2016), and medicine (Bycroft et al., 2018), and perhaps rarer in fields such as sociology (Camerer et al., 2018) and psychology (Forscher et al., 2020; Frank et al., 2017; Makel et al., 2012). They are also rare in neuroscience, a field that faces a reproducibility crisis (Baker, 2016; Botvinik-Nezer et al., 2020; Button et al., 2013).

Reproducibility has been a particular concern for measurements of mouse behavior (Kafkafi et al., 2018). Although the methods can be generally reproduced across laboratories, the results can be surprisingly different (‘methods reproducibility’ vs. ‘results reproducibility’, Goodman et al., 2016). Even seemingly simple assays of responses to pain or stress can be swayed by extraneous factors (Chesler et al., 2002; Crabbe et al., 1999) such as the sex of the experimenter (Sorge et al., 2014). Behavioral assays can be difficult to reproduce across laboratories even when they share a similar apparatus (Chesler et al., 2002; Crabbe et al., 1999; Sorge et al., 2014). This difficulty is not simply due to genetic variation: behavioral variability is as large in inbred mice as in outbred mice (Tuttle et al., 2018).

Difficulties in reproducing mouse behavior across laboratories would hinder the increasing number of studies that investigate decision making in mice. Physiological studies of decision making are increasingly carried out in mice to access the unrivaled arsenal of genetic, imaging, and physiological tools available for mouse brains (Carandini and Churchland, 2013; Glickfeld et al., 2014; O'Connor et al., 2009). Our collaboration, International Brain Laboratory, 2017, aims to leverage these approaches by exploring the neural basis of the same mouse behavior in multiple laboratories. It is thus crucial for this endeavor that the relevant behavioral assays be reproducible both in methods and in results.

Studying decision-making requires a task that places specific sensory, cognitive, and motor demands over hundreds of trials, affording strong constraints to behavior. The task should be complex enough to expose the neural computations that support decision-making but simple enough for mice to learn, and easily extendable to study further aspects of perception and cognition. Moreover, it can be invaluable to have ready access to the brain for neural recordings and manipulations, a consideration that favors tasks that involve head fixation.

To meet these criteria, we adopted a modified version of the classical ‘two-alternative forced-choice’ perceptual detection task. In the classical task, the subject indicates the position of a stimulus that can be in one of two positions with equal probability (e.g. Carandini and Churchland, 2013; Tanner and Swets, 1954). In the modified version, the probability of the stimulus being in one position changes over time (Terman and Terman, 1972). This change in probability may affect sensory decisions by directing spatial attention (Cohen and Maunsell, 2009; Liston and Stone, 2008) and by biasing the decision process (Hanks et al., 2011). It modifies the expected value of the choices, echoing changes in reward probability or size, which affect perceptual choices (e.g. Feng et al., 2009; Whiteley and Sahani, 2008) and drive value-based choices (Corrado et al., 2005; Fan et al., 2018; Herrnstein, 1961; Lau and Glimcher, 2005; Miller et al., 2019).

In the task, mice detect the presence of a visual grating to their left or right, and report the perceived location with a simple movement: by turning a steering wheel (Burgess et al., 2017). The task difficulty is controlled by varying the contrast across trials. The reward for a correct response is a drop of sugar water that is not contingent on licking the spout. The probability of stimulus appearance at the two locations is asymmetric and changes across blocks of trials. Mice thus make decisions by using both their vision and their recent experience. When the visual stimulus is evident (contrast is high), they should mostly use vision, and when the visual stimulus is ambiguous (contrast is low or zero), they should consider prior information (Whiteley and Sahani, 2008) and choose the side that has recently been more likely.

We here present results from a large cohort of mice trained in the task, demonstrating reproducible methods and reproducible results across laboratories. In all laboratories, most mice learned the task, although often at a different pace. After learning they performed the task in a comparable manner, and with no significant difference across laboratories. Mice in different laboratories adopted a comparable reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability.

To facilitate reuse and reproducibility, we adopt an open science approach: we describe and share the hardware and software components and the experimental protocols. It is increasingly recognized that data and techniques should be made fully available to the broader community (Beraldo et al., 2019; Charles et al., 2020; Forscher et al., 2020; Koscielny et al., 2014; Poldrack and Gorgolewski, 2014; de Vries et al., 2020). Following this approach, we established an open-access data architecture pipeline (Bonacchi et al., 2020) and use it to release the >5 million mouse choices at data.internationalbrainlab.org. These results reveal that a complex mouse behavior can be successfully reproduced across laboratories, enabling collaborative studies of brain function in behaving mice.

To train mice consistently within and across laboratories we developed a standardized training pipeline (Figure 1a). First, we performed surgery to implant a headbar for head-fixation (IBL Protocol for headbar implant surgery in mice [The International Brain Laboratory, 2020a]). During the subsequent recovery period, we handled the mice and weighed them daily. Following recovery, we put mice on water control and habituated them to the experimental setup (IBL Protocol for mice training [The International Brain Laboratory, 2020b]). Throughout these steps, we checked for adverse effects such as developing a cataract during surgery or pain after surgery or substantial weight loss following water control (four mice excluded out of 210) (Guo et al., 2014).

Figure 1 with 2 supplements see all

Download asset Open asset

Mice were then trained in two stages: first they learned a basic task (Burgess et al., 2017), where the probability of a stimulus appearing on the left or the right was equal (50:50), and then they learned the full task, where the probability of stimuli appearing on the left vs. right switched in blocks of trials between 20:80 and 80:20. Out of 206 mice that started training, 140 achieved proficiency in the basic task, and 98 in the full task (see Appendix 1—table 1d for training progression and proficiency criteria). The basic task is purely perceptual: the only informative cue is the visual stimulus. The full task instead invites integration of perception with recent experience: when stimuli are ambiguous it is best to choose the more likely option.

To facilitate reproducibility, we standardized multiple variables, measured multiple other variables, and shared the behavioral data in a database that we inspected regularly. By providing standards and guidelines, we sought to control variables such as mouse strain and provider, age range, weight range, water access, food protein, and fat. We did not attempt to standardize other variables such as light-dark cycle, temperature, humidity, and environmental sound, but we documented and measured them regularly (Appendix 1—table 1a; Voelkl et al., 2020). Data were shared and processed across the collaboration according to a standardized web-based pipeline (Bonacchi et al., 2020). This pipeline included a colony management database that stored data about each session and mouse (e.g. session start time, animal weight, etc.), a centralized data repository for files generated in the task (e.g. behavioral responses and compressed video and audio files), and a platform which provided automated analyses and daily visualizations (data.internationalbrainlab.org) (Yatsenko et al., 2018).

Mice were trained in a standardized setup involving a steering wheel placed in front of a screen (Figure 1b,c). The visual stimulus, a grating, appeared at variable contrast on the left or right half of the screen. The stimulus position was coupled with movements of the response wheel, and mice indicated their choices by turning the wheel left or right to bring the grating to the center of the screen (Burgess et al., 2017). Trials began after the mouse held the wheel still for 0.4–0.7 s and were announced by an auditory ‘go cue’. Correct decisions were rewarded with sweetened water (10% sucrose solution), whereas incorrect decisions were indicated by a noise burst and were followed by a longer inter-trial interval (2 s) (Guo et al., 2014, Figure 1—figure supplement 1). The experimental setups included systems for head-fixation, visual and auditory stimuli presentation, and recording of video and audio (Figure 1c). These were standardized and based on open-source hardware and software (IBL protocol for setting up the behavioral training rig [The International Brain Laboratory, 2021a]).

Training succeeded but with different rates across mice and laboratories

The training procedures succeeded in all laboratories, but the duration of training varied across mice and laboratories (Figure 2). There was substantial variation among mice, with the fastest learner achieving basic task proficiency in 3 days, and the slowest after 59 days (Figure 2a). Estimates of contrast threshold did not vary much during training (Figure 2b), with an interquartile range that slightly decreased from 12–29% during the first 10 training days, and 11–23% thereafter. Likewise, bias did not vary much, with an interquartile range of −5.2% to 7.1% during the first 10 days to −5.6% to 6.5% thereafter (Figure 2c). These effects were similar across laboratories (Figure 2e,f). The average training took 18.4 ± 13.0 days (s.d., n = 140, Figure 2g). The number of days needed to achieve basic task proficiency was different across laboratories (Figure 2g, p<0.001, Kruskal-Wallis nonparametric test followed by a post-hoc Dunn’s multiple comparisons test). Some labs had homogeneous learning rates (e.g. Lab two within-lab interquartile range of 8 days), while other labs had larger variability (e.g. Lab six interquartile range of 22 days).

Figure 2 with 2 supplements see all

Download asset Open asset

These differences in learning rates across laboratories could not be explained by differences in number of trials per session. To account for these possible differences, we measured training proficiency across trials rather than days (Figure 2—figure supplement 1). For mice that learned the task, the average training took 10.8 ± 8.6 thousands of trials (s.d., n = 140), similar to the 13 thousand trials of the example mouse from Lab 1 (Figure 2a, black). The fastest learner met training criteria in one thousand trials, the slowest 43 thousand trials. To reach 80% on easy trials took on average ~7.4 thousand trials.

Variability in learning rates was also not due to systematic differences in choice bias or visual sensitivity (Figure 2b–c). Mice across laboratories were not systematically biased towards a particular side and the absolute bias was similar across laboratories (9.8 ± 12.1 [s.d.] on average). Likewise, measures of contrast threshold stabilized after the first ~10 sessions, which was on average 17.8 ± 11.7% on average across institutions.

The variability in performance across mice decreased as training progressed, but it did not disappear (Figure 2—figure supplement 2). Variability in performance was larger in the middle of training than towards the end. For example, between training days 15 and 40, when the average performance on easy trials of the mice increased from 80.7% to 91.1%, the variation across mice (s.d.) of performance on easy trials decreased from 19.1% to 10.1%.

To some extent, a mouse’s performance in the first five sessions predicted how long it would take the mouse to become proficient. A Random Forests decoder applied to change in performance (% correct with easy, high-contrast stimuli) in the first five sessions was able to predict whether a mouse would end up in the bottom quartile of learning speed (the slowest learners) with accuracy of 53% (where chance is 25%). Conversely, the chance of misclassifying a fast-learning, top quartile mouse as a slow-learning, bottom quartile mouse, was only 7%.

Overall, our procedures succeeded in training tens of mice in each laboratory (Figure 2h). Indeed, there was a 80% probability that mice would learn the task within the 40 days that were usually allotted, and when mice were trained for a longer duration, the success rate rose even further (Figure 2h). There was, however, variability in learning rates across mice and laboratories. This variability is intriguing and would present a challenge for projects that aim to study learning. We next ask whether the behavior of trained mice was consistent and reproducible across laboratories.

Performance in the basic task was indistinguishable across laboratories

Once mice achieved basic task proficiency, multiple measures of performance became indistinguishable across laboratories (Figure 3a–e). We first examined the psychometric curves for the three sessions leading up to proficiency, which showed a stereotypical shape across mice and laboratories (Figure 3a). The average across mice of these psychometric curves was similar across laboratories (Figure 3b). The lapse rates (i.e., the errors made in response to easy contrasts of 50% and 100%) were low (9.5 ± 3.6%, Figure 3c) with no significant difference across laboratories. The slope of the curves, which measures contrast sensitivity, was also similar across laboratories, at 14.3 ± 3.8 (s.d., n = 7 laboratories, Figure 3d). Finally, the horizontal displacement of the curve, which measures response bias, was small at 0.3 ± 8.4 (s.d., n = 7, Figure 3e). None of these measures showed a significant difference across laboratories, either in median (performance: p=0.63, threshold: p=0.81, bias: p=0.81, FDR corrected Kruskal-Wallis test) or in variance (performance: p=0.09, threshold: p=0.57, bias: p=0.57, FDR corrected Levene’s test). Indeed, mouse choices were no more consistent within labs than across labs (Figure 3—figure supplement 1a).

Figure 3 with 5 supplements see all

Download asset Open asset

Variations across laboratories were also small in terms of trial duration and number of trials per session, even though we had made no specific effort to harmonize these variables. The median time from stimulus onset to feedback (trial duration, a coarse measure of reaction time) was 468 ± 221 ms, showing some differences across laboratories (Figure 3—figure supplement 2a, p=0.004, Kruskal-Wallis nonparametric test). Mice on average performed 719 ± 223 trials per session (Figure 3—figure supplement 2b). This difference was significant (p<10⁻⁶, one-way ANOVA) but only in one laboratory relative to the rest.

Variation in performance across laboratories was so low that we were not able to assign a mouse to a laboratory based on performance (Figure 3f). Having found little variation in behavioral variables when considered one by one (Figure 3c–e), we asked whether mice from different laboratories may exhibit characteristic combinations of these variables. We thus trained a Naive Bayes classifier (Pedregosa et al., 2011), with 2000 random sub-samples of eight mice per laboratory, to predict lab membership from these behavioral variables. First, we established a positive control, checking that the classifier showed the expected behavior when provided with an informative variable: the time zone in which animals were trained. If the classifier was given this variable during training and testing it performed above chance (Figure 3f, Positive control) and the confusion matrix showed clusters of labs located in the same time zone (Figure 3—figure supplement 3b). Next, we trained and tested the classifier on a null distribution obtained by shuffling the lab labels, and as expected we found that the classifier was at chance (Figure 3f, Shuffle). Finally, we trained and tested the classifier on the behavioral data, and found that it performed at chance level, failing to identify the laboratory of origin of the mice (Figure 3f, Mouse behavior). Its mean accuracy was 0.12 ± 0.051, and a 95th percentile of its distribution included the chance level of 0.11 (mean of the null distribution). The classifications were typically off-diagonal in the confusion matrix, and hence incorrect (Figure 3—figure supplement 3c). Similar results were obtained with two other classifier algorithms (Figure 3—figure supplement 3d–i).

This consistency across laboratories was not confined to the three sessions that led to proficiency, and was observed both in earlier and in later sessions. We repeated our analyses for three sessions that led to mice achieving a looser definition of proficiency: a single criterion of 80% correct for easy stimuli, without criteria on response bias and contrast threshold. In these three sessions, which could be earlier but not later than the ones analyzed in Figure 3, mouse behavior was again consistent across laboratories: a decoder failed to identify the origin of a mouse based on its behavioral performance (Figure 3—figure supplement 4). Finally, we repeated our analysis for sessions obtained after the mouse achieved proficiency in the basic task. We selected the first three sessions of the full task, and again found no significant difference across laboratories: mice had similar performance at high contrast, similar contrast threshold, and similar response bias across laboratories (Figure 3—figure supplement 5).

Performance in the full task was indistinguishable across laboratories

After training the mice in the purely sensory, basic task, we introduced them to the full task, where optimal performance requires integration of sensory perception with recent experience (Figure 4a,b). Specifically, we introduced block-wise biases in the probability of stimulus location, and therefore the more likely correct choice. Sessions started with a block of unbiased trials (50:50 probability of left vs. right) and then alternated between blocks of variable length (20–100 trials) biased toward the right (20:80 probability) or toward the left (80:20 probability) (Figure 4a). In these blocks, the probability of 0% contrast stimuli was doubled to match the probability of other contrasts. The transition between blocks was not signaled, so the mice had to estimate a prior for stimulus location based on recent task statistics. This task invites the mice to integrate information across trials and to use this prior knowledge in their perceptual decisions.

Figure 4

Download asset Open asset

To assess how mice used information about block structure, we compared their psychometric curves in the different block types (Figure 4b,c). Mice incorporated block priors into their choices already from the first sessions in which they were exposed to this full task (Figure 3—figure supplement 5a). To assess proficiency in the full task, we used a fixed set of criteria (Figure 1—figure supplement 1d). We considered performance in the three sessions in which mice reached proficiency on the full task (Figure 1—figure supplement 1d). For mice that reached full task proficiency, the average training from start to finish took 31.5 ± 16.1 days, or 20,494 ± 10,980 trials (s.d., n = 98). The example mouse from Lab 1 (Figure 2a, black) took 19 days. The fastest learner achieved proficiency in 8 days (4691 trials), the slowest 81 days (61,316 trials). The curves for the 20:80 and 80:20 blocks were shifted relative to the curve for the 50:50 block, with mice more likely to choose right in the 20:80 blocks (where right stimuli appeared 80% of the time) and to choose left in the 80:20 blocks (where left stimuli appeared 80% of the time). As expected, block structure had the greatest impact on choices when sensory evidence was absent (contrast = 0%, Figure 4c). In these conditions, it makes sense for the mice to be guided by recent experience, and thus to choose differently depending on the block prior.

Changes in block type had a similar effect on mice in all laboratories (Figure 4d,e). The average shift in rightward choices was invariably highest at 0% contrast, where rightward choices in the two blocks differed by an average of 28.5%. This value did not significantly differ across laboratories any more than it differed within laboratories (one-way ANOVA F(6) = 1.345, p=0.2455, Figure 4d,e).

An analysis of the psychometric curves showed highly consistent effects of block type, with no significant differences across laboratories (Figure 4f–i). Changes in block type did not significantly affect the contrast threshold (Wilcoxon Signed-Rank test, p=0.85, n = 98 mice, Figure 4f). However, it did change the lapse rates, which were consistently higher on the left in the 20:80 blocks and on the right in the 80:20 blocks (Wilcoxon Signed-Rank test, lapse left; p<10⁻⁶, lapse right; p<10⁻⁷, Figure 4g,h). Finally, as expected there was a highly consistent change in overall bias, with curves shifting to the left in 20:80 trials and to the right in 80:20 trials (Wilcoxon Signed-Rank test, p<10⁻¹⁶, Figure 4i). Just as in the basic task (Figure 3), a classifier trained on these variables could not predict above chance the origin laboratory of individual mice above chance (Figure 4j). Moreover, neither choice fractions nor bias shifts showed within-lab consistency that was larger than expected by chance (Figure 3—figure supplement 1b,c). This confirms that mice performed the full task similarly across laboratories.

A probabilistic model reveals common strategies across mice and laboratories

Lastly, we investigated the strategies used by the mice in the basic task and in the full task and asked whether these strategies were similar across mice and laboratories. Mice might incorporate non-sensory information into their decisions, even when such information is not predictive of reward (Busse et al., 2011; Lak et al., 2020a). Therefore, even when reaching comparable performance levels, different mice might be weighing task variables differently when making a decision.

To quantify how different mice form their decisions, we used a generalized linear model (Figure 5a). The model is based on similar approaches used in both value-based (Lau and Glimcher, 2005) and sensory-based decision making (Busse et al., 2011; Pinto et al., 2018). In the model, the probability of making a right choice is calculated from a logistic function of the linear weighted sum of several predictors: the stimulus contrast, the outcome of the previous trial, and a bias term that represents the overall preference of a mouse for a particular choice across sessions (Busse et al., 2011). In the case of the full task, we added a parameter representing the identity of the block, measuring the weight of the prior stimulus statistics in the mouse decisions (Figure 5a).

Figure 5 with 2 supplements see all

Download asset Open asset

We fitted the model to the choices of each individual mouse over three sessions by logistic regression. The mean condition number for the basic model was 2.4 and 3.2 for the full model. The low condition numbers for both models do not suffer from multicollinearity and the coefficients are therefore interpretable.

The model fit the mouse choices well and captured the relative importance of sensory and non-sensory information across mice and laboratories (Figure 5b–g). The model was able to accurately predict the behavior of individual mice, both in the basic task (Figure 5b) and in the full task (Figure 5e). As expected, visual terms had large weights, which grew with contrast to reach values above 3 at 100% contrast. Weights for non-sensory factors were much lower (Figure 5c,f, note different scale). Weights for past choices were positive for both rewarded (basic 0.19, full 0.42) and unrewarded previous trials (basic 0.33, full 0.46), suggesting that mice were perseverant in their choice behavior (Figure 5d,g; Figure 5—figure supplement 1a). Indeed, previous choices more strongly influenced behavior in the full task, both after rewarded (t(125) = 10.736, p<10⁻⁶) and unrewarded trials (t(125) = 4.817, p<10⁻⁶). Importantly, fitted weights and the model’s predictive accuracy (unbiased: 81.03 + 5.1%, biased 82.1 + 5.8%) (Figure 5—figure supplement 2) were similar across laboratories, suggesting an overall common strategy.

The model coefficients demonstrated that mice were perseverant in their actions (Figure 5d,g; Figure 5—figure supplement 1a). This behavior can arise from insensitivity to the outcome of previous trials and from slow drifts in the decision process across trials, arising from correlations in past choices independently of reward (Lak et al., 2020a; Mendonça et al., 2018). To disentangle these two factors, we corrected for slow across-trial drifts in the decision process (Lak et al., 2020a). This correction revealed a win-stay/lose-switch strategy in both the basic and full task, which coexists with slow drifts in choice bias across trials (Figure 5—figure supplement 1b). Moreover, the dependence on history- was modulated by confidence in the previous trial Figure 5—figure supplement 1c (Lak et al., 2020a; Mendonça et al., 2018). These effects were generally consistent across laboratories.

These results reveal that a complex mouse behavior can be successfully reproduced across laboratories, and more generally suggest a path toward improving reproducibility in neuroscience. To study mouse behavior across laboratories, we developed and implemented identical experimental equipment and a standard set of protocols. Not only did mice learn the task in all laboratories, but critically, after learning they performed the task comparably across laboratories. Mice in different laboratories had similar psychophysical performance in a purely sensory version of the task and adopted similar choice strategies in the full task, where they benefited from tracking the stimulus prior probability. Behavior showed variations across sessions and across mice, but these variations were no larger across laboratories than within laboratories.

Success did not seem guaranteed at the outset, because neuroscience faces a crisis of reproducibility (Baker, 2016; Botvinik-Nezer et al., 2020; Button et al., 2013) particularly when it comes to measurements of mouse behavior (Chesler et al., 2002; Crabbe et al., 1999; Kafkafi et al., 2018; Sorge et al., 2014; Tuttle et al., 2018). To solve this crisis, three solutions have been proposed: large studies, many teams, and upfront registration (Ioannidis, 2005). Our approach incorporates all three of these solutions. First, we collected vast amounts of data: 5 million choices from 140 mice. Second, we involved many teams, obtaining data in seven laboratories in three countries. Third, we standardized the experimental protocols and data analyses upfront, which is a key component of pre-registration.

An element that may have contributed to success is the collaborative, open-science nature of our initiative (Wool and International Brain Laboratory, 2020). Open-science collaborative approaches are increasingly taking hold in neuroscience (Beraldo et al., 2019; Charles et al., 2020; Forscher et al., 2020; Koscielny et al., 2014; Poldrack and Gorgolewski, 2014; de Vries et al., 2020). Our work benefited from collaborative development of the behavioral assay, and from frequent and regular meetings where data were reviewed across laboratories (Figure 6). These meetings helped identify problems at the origin, provide immediate feedback, and find collective solutions. Moreover, our work benefited from constant efforts at standardization. We took great care in standardizing and documenting the behavioral apparatus and the training protocol (see Appendices), to facilitate implementation across our laboratories and to encourage wider adoption by other laboratories. The protocols, hardware designs and software code are open-source and modular, allowing adjustments to accommodate a variety of scientific questions. The data are accessible at data.internationalbrainlab.org, and include all >5 million choices made by the mice.

Figure 6

Download asset Open asset

Another element that might have contributed to success is our choice of behavioral task, which places substantial requirements on the mice while not being too complex. Previous failures to reproduce mouse behavior across laboratories typically arose in studies of unconstrained behavior such as responses to pain or stress (Chesler et al., 2002; Crabbe et al., 1999; Kafkafi et al., 2018; Sorge et al., 2014; Tuttle et al., 2018). Operant behaviors may be inherently more reproducible than the assays used in these studies. To be able to study decision making, and in hopes of achieving reproducibility, we designed a task that engages multiple brain processes from sensory perception and integration of evidence to combination of priors and evidence. It seems likely that reproducibility is easier to achieve if the task requirements are substantial (so there is less opportunity to engage in other behaviors) but not so complex that they fail to motivate and engage. Tasks that are too simple and unconstrained or too arbitrary and difficult may be hard to reproduce.

There are of course multiple ways to improve on our results, for example by clarifying, and if desired resolving, the differences in learning rate across mice, both within and across laboratories. The learning rate is a factor that we had not attempted to control, and we cannot here ascertain the causes of its variability. We suspect that it might arise partly from variations in the expertise and familiarity of different labs with visual neuroscience and mouse behavior, which may impede standardization. If so, perhaps as experimenters gain further experience the differences in learning times will decrease. Indeed, an approach to standardizing learning rates might be to introduce full automation in behavioral training by reducing or even removing the need for human intervention (Aoki et al., 2017; Poddar et al., 2013; Scott et al., 2015). Variability in learning rates may be further reduced by individualized, dynamic training methods (Bak et al., 2016). If desired, such methods could also be aimed at obtaining uniform numbers of trials and reaction times across laboratories.

It would be fruitful to characterize the behavior beyond the turning of the wheel, analyzing the movement of the limbs. Here we only analyzed whether the wheel was turned to the left or to the right, but to turn the wheel the mice move substantial parts of their body, and they do so in diverse ways (e.g. some use two hands, others use one, and so on). Through videography (Mathis et al., 2018) one could classify these movements and perhaps identify behavioral strategies that provide insight in the performance of the task and into the diversity of behavior that we observed across sessions and across mice.

We hope that this large, curated dataset will serve as a benchmark for testing better models of decision-making. The model of choice that we used here, based on stimuli, choice history, reward history, and bias, is only a starting point (Busse et al., 2011). Indeed, the ‘perseverance’ that we represented by the reward history weights can arise from slow fluctuations in decision process over trials (Lak et al., 2020a; Mendonça et al., 2018) and history-dependence is modulated by confidence in the previous trial (Lak et al., 2020b; Lak et al., 2020a; Urai et al., 2017). In addition, our data can help investigate phenomena such as the origin of lapses (Ashwood et al., 2020; Pisupati et al., 2021), the tracking of changes between prior blocks (Norton et al., 2019), the effect of fluctuating engagement states (McGinley et al., 2015), and the dynamics of trial-by-trial learning (Roy et al., 2021).

To encourage and support community adoption of this task, we provide detailed methods and protocols. These methods and protocols provide a path to training mice in this task and reproducing their behavior across laboratories, but of course we cannot claim that they constitute the optimal path. In designing our methods, we made many choices that were based on intuition rather than rigorous experimentation. We don’t know what is crucial in these methods and what is not.

The reproducibility of this mouse behavior makes it a good candidate for studies of the brain mechanisms underlying decision making. A reproducible behavioral task can be invaluable to establish the neural basis of behavior. If different studies use the same task, they can directly compare their findings. There are indeed illustrious examples of behavioral tasks that serve this role. For studying decision-making in primates, these include the tactile flutter comparison task (de Lafuente and Romo, 2005; Romo et al., 2012) and the random dots visual discrimination task (Ding and Gold, 2013; Newsome et al., 1989; Shadlen and Kiani, 2013). Both tasks have been used in multiple studies to record from different brain regions while enabling a meaningful comparison of the results. Conversely, without a standardized behavioral task we face the common situation where different laboratories record from different neurons in different regions in different tasks, likely drawing different conclusions and likely not sharing their data. In that situation it is not possible to establish which factors determine the different conclusions and come to a collective understanding.

Now that we have developed this task and established its reproducibility across laboratories, the International Brain Laboratory is using it together with neural recordings, which are performed in different laboratories and combined into a single large data set. Other laboratories that adopt this task for studies of neural function will then be able to rely on this large neural dataset to complement their more focused results. Moreover, they will be able to compare each other’s results, knowing that any difference between them is unlikely to be due to differences in behavior. We also hope that these resources catalyze the development of new adaptations and variations of our approach, and accelerate the use of mice in high quality, reproducible studies of neural correlates of decision-making.

All procedures and experiments were carried out in accordance with the local laws and following approval by the relevant institutions: the Animal Welfare Ethical Review Body of University College London; the Institutional Animal Care and Use Committees of Cold Spring Harbor Laboratory, Princeton University, and University of California at Berkeley; the University Animal Welfare Committee of New York University; and the Portuguese Veterinary General Board.

Data for all figures is available at https://data.internationalbrainlab.org/.

1. 1. Abdalla H
   2. Abramowski A
   3. Aharonian F
   4. Ait Benkhali F
   5. Angüner EO
   6. Arakawa M
   7. Arrieta M
   8. Aubert P
   9. Backes M
   10. Balzer A
   11. Barnard M
   12. Becherini Y
   13. Becker Tjus J
   14. Berge D
   15. Bernhard S
   16. Bernlöhr K
   17. Blackwell R
   18. Böttcher M
   19. Boisson C
   20. Bolmont J
   21. Bonnefoy S
   22. Bordas P
   23. Bregeon J
   24. Brun F
   25. Brun P
   26. Bryan M
   27. Büchele M
   28. Bulik T
   29. Capasso M
   30. Carrigan S
   31. Caroff S
   32. Carosi A
   33. Casanova S
   34. Cerruti M
   35. Chakraborty N
   36. Chaves RCG
   37. Chen A
   38. Chevalier J
   39. Colafrancesco S
   40. Condon B
   41. Conrad J
   42. Davids ID
   43. Decock J
   44. Deil C
   45. Devin J
   46. deWilt P
   47. Dirson L
   48. Djannati-Ataï A
   49. Domainko W
   50. Donath A
   51. Drury LOC
   52. Dutson K
   53. Dyks J
   54. Edwards T
   55. Egberts K
   56. Eger P
   57. Emery G
   58. Ernenwein J-P
   59. Eschbach S
   60. Farnier C
   61. Fegan S
   62. Fernandes MV
   63. Fiasson A
   64. Fontaine G
   65. Förster A
   66. Funk S
   67. Füßling M
   68. Gabici S
   69. Gallant YA
   70. Garrigoux T
   71. Gast H
   72. Gaté F
   73. Giavitto G
   74. Giebels B
   75. Glawion D
   76. Glicenstein JF
   77. Gottschall D
   78. Grondin M-H
   79. Hahn J
   80. Haupt M
   81. Hawkes J
   82. Heinzelmann G
   83. Henri G
   84. Hermann G
   85. Hinton JA
   86. Hofmann W
   87. Hoischen C
   88. Holch TL
   89. Holler M
   90. Horns D
   91. Ivascenko A
   92. Iwasaki H
   93. Jacholkowska A
   94. Jamrozy M
   95. Jankowsky D
   96. Jankowsky F
   97. Jingo M
   98. Jouvin L
   99. Jung-Richardt I
   100. Kastendieck MA
   101. Katarzyński K
   102. Katsuragawa M
   103. Katz U
   104. Kerszberg D
   105. Khangulyan D
   106. Khélifi B
   107. King J
   108. Klepser S
   109. Klochkov D
   110. Kluźniak W
   111. Komin N
   112. Kosack K
   113. Krakau S
   114. Kraus M
   115. Krüger PP
   116. Laffon H
   117. Lamanna G
   118. Lau J
   119. Lees J-P
   120. Lefaucheur J
   121. Lemière A
   122. Lemoine-Goumard M
   123. Lenain J-P
   124. Leser E
   125. Lohse T
   126. Lorentz M
   127. Liu R
   128. López-Coto R
   129. Lypova I
   130. Marandon V
   131. Malyshev D
   132. Marcowith A
   133. Mariaud C
   134. Marx R
   135. Maurin G
   136. Maxted N
   137. Mayer M
   138. Meintjes PJ
   139. Meyer M
   140. Mitchell AMW
   141. Moderski R
   142. Mohamed M
   143. Mohrmann L
   144. Morå K
   145. Moulin E
   146. Murach T
   147. Nakashima S
   148. de Naurois M
   149. Ndiyavala H
   150. Niederwanger F
   151. Niemiec J
   152. Oakes L
   153. O’Brien P
   154. Odaka H
   155. Ohm S
   156. Ostrowski M
   157. Oya I
   158. Padovani M
   159. Panter M
   160. Parsons RD
   161. Paz Arribas M
   162. Pekeur NW
   163. Pelletier G
   164. Perennes C
   165. Petrucci P-O
   166. Peyaud B
   167. Piel Q
   168. Pita S
   169. Poireau V
   170. Poon H
   171. Prokhorov D
   172. Prokoph H
   173. Pühlhofer G
   174. Punch M
   175. Quirrenbach A
   176. Raab S
   177. Rauth R
   178. Reimer A
   179. Reimer O
   180. Renaud M
   181. de los Reyes R
   182. Rieger F
   183. Rinchiuso L
   184. Romoli C
   185. Rowell G
   186. Rudak B
   187. Rulten CB
   188. Safi-Harb S
   189. Sahakian V
   190. Saito S
   191. Sanchez DA
   192. Santangelo A
   193. Sasaki M
   194. Schandri M
   195. Schlickeiser R
   196. Schüssler F
   197. Schulz A
   198. Schwanke U
   199. Schwemmer S
   200. Seglar-Arroyo M
   201. Settimo M
   202. Seyffert AS
   203. Shafi N
   204. Shilon I
   205. Shiningayamwe K
   206. Simoni R
   207. Sol H
   208. Spanier F
   209. Spir-Jacob M
   210. Stawarz Ł.
   211. Steenkamp R
   212. Stegmann C
   213. Steppa C
   214. Sushch I
   215. Takahashi T
   216. Tavernet J-P
   217. Tavernier T
   218. Taylor AM
   219. Terrier R
   220. Tibaldo L
   221. Tiziani D
   222. Tluczykont M
   223. Trichard C
   224. Tsirou M
   225. Tsuji N
   226. Tuffs R
   227. Uchiyama Y
   228. van der Walt DJ
   229. van Eldik C
   230. van Rensburg C
   231. van Soelen B
   232. Vasileiadis G
   233. Veh J
   234. Venter C
   235. Viana A
   236. Vincent P
   237. Vink J
   238. Voisin F
   239. Völk HJ
   240. Vuillaume T
   241. Wadiasingh Z
   242. Wagner SJ
   243. Wagner P
   244. Wagner RM
   245. White R
   246. Wierzcholska A
   247. Willmann P
   248. Wörnlein A
   249. Wouters D
   250. Yang R
   251. Zaborov D
   252. Zacharias M
   253. Zanin R
   254. Zdziarski AA
   255. Zech A
   256. Zefi F
   257. Ziegler A
   258. Zorn J
   259. Żywucka N

   (2018) The H.E.S.S. galactic plane survey

   Astronomy & Astrophysics 612:201732098.

   https://doi.org/10.1051/0004-6361/201732098

   • Google Scholar
2. 1. Aoki R
   2. Tsubota T
   3. Goya Y
   4. Benucci A

   (2017) An automated platform for high-throughput mouse behavior and physiology with voluntary head-fixation

   Nature Communications 8:1–9.

   https://doi.org/10.1038/s41467-017-01371-0

   • PubMed
   • Google Scholar
3. Preprint

   1. Ashwood ZC
   2. Roy NA
   3. Stone IR
   4. Churchland AK
   5. Pouget A
   6. Pillow JW

   (2020) Mice alternate between discrete strategies during perceptual decision-making

   bioRxiv.

   https://doi.org/10.1101/2020.10.19.346353

   • Google Scholar
4. Book

   1. Bak JH
   2. Choi JY
   3. Akrami A
   4. Witten I
   5. Pillow JW
   6. Sugiyama M
   7. Luxburg UV
   8. Guyon I

   (2016)

   Adaptive optimal training of animal behavior

   In: Lee D. D, Garnett R, editors. Advances in Neural Information Processing Systems. Curran Associates, Inc. pp. 1947–1955.

   • Google Scholar
5. 1. Baker M

   (2016) 1,500 scientists lift the lid on reproducibility

   Nature 533:452–454.

   https://doi.org/10.1038/533452a

   • PubMed
   • Google Scholar
6. 1. Beraldo FH
   2. Palmer D
   3. Memar S
   4. Wasserman DI
   5. Lee WV
   6. Liang S
   7. Creighton SD
   8. Kolisnyk B
   9. Cowan MF
   10. Mels J
   11. Masood TS
   12. Fodor C
   13. Al-Onaizi MA
   14. Bartha R
   15. Gee T
   16. Saksida LM
   17. Bussey TJ
   18. Strother SS
   19. Prado VF
   20. Winters BD
   21. Prado MA

   (2019) MouseBytes, an open-access high-throughput pipeline and database for rodent touchscreen-based cognitive assessment

   eLife 8:e49630.

   https://doi.org/10.7554/eLife.49630

   • PubMed
   • Google Scholar
7. Preprint

   1. Bonacchi N
   2. Chapuis G
   3. Churchland AK
   4. Harris KD
   5. Hunter M
   6. Rossant C
   7. Sasaki M
   8. Shen S
   9. Steinmetz NA
   10. Walker EY
   11. Winter O
   12. Wells M
   13. The International Brain Laboratory

   (2020) Data architecture and visualization for a large-scale neuroscience collaboration

   bioRxiv.

   https://doi.org/10.1101/827873

   • Google Scholar
8. 1. Botvinik-Nezer R
   2. Holzmeister F
   3. Camerer CF
   4. Dreber A
   5. Huber J
   6. Johannesson M
   7. Kirchler M
   8. Iwanir R
   9. Mumford JA
   10. Adcock RA
   11. Avesani P
   12. Baczkowski BM
   13. Bajracharya A
   14. Bakst L
   15. Ball S
   16. Barilari M
   17. Bault N
   18. Beaton D
   19. Beitner J
   20. Benoit RG
   21. Berkers R
   22. Bhanji JP
   23. Biswal BB
   24. Bobadilla-Suarez S
   25. Bortolini T
   26. Bottenhorn KL
   27. Bowring A
   28. Braem S
   29. Brooks HR
   30. Brudner EG
   31. Calderon CB
   32. Camilleri JA
   33. Castrellon JJ
   34. Cecchetti L
   35. Cieslik EC
   36. Cole ZJ
   37. Collignon O
   38. Cox RW
   39. Cunningham WA
   40. Czoschke S
   41. Dadi K
   42. Davis CP
   43. Luca A
   44. Delgado MR
   45. Demetriou L
   46. Dennison JB
   47. Di X
   48. Dickie EW
   49. Dobryakova E
   50. Donnat CL
   51. Dukart J
   52. Duncan NW
   53. Durnez J
   54. Eed A
   55. Eickhoff SB
   56. Erhart A
   57. Fontanesi L
   58. Fricke GM
   59. Fu S
   60. Galván A
   61. Gau R
   62. Genon S
   63. Glatard T
   64. Glerean E
   65. Goeman JJ
   66. Golowin SAE
   67. González-García C
   68. Gorgolewski KJ
   69. Grady CL
   70. Green MA
   71. Guassi Moreira JF
   72. Guest O
   73. Hakimi S
   74. Hamilton JP
   75. Hancock R
   76. Handjaras G
   77. Harry BB
   78. Hawco C
   79. Herholz P
   80. Herman G
   81. Heunis S
   82. Hoffstaedter F
   83. Hogeveen J
   84. Holmes S
   85. Hu CP
   86. Huettel SA
   87. Hughes ME
   88. Iacovella V
   89. Iordan AD
   90. Isager PM
   91. Isik AI
   92. Jahn A
   93. Johnson MR
   94. Johnstone T
   95. Joseph MJE
   96. Juliano AC
   97. Kable JW
   98. Kassinopoulos M
   99. Koba C
   100. Kong XZ
   101. Koscik TR
   102. Kucukboyaci NE
   103. Kuhl BA
   104. Kupek S
   105. Laird AR
   106. Lamm C
   107. Langner R
   108. Lauharatanahirun N
   109. Lee H
   110. Lee S
   111. Leemans A
   112. Leo A
   113. Lesage E
   114. Li F
   115. Li MYC
   116. Lim PC
   117. Lintz EN
   118. Liphardt SW
   119. Losecaat Vermeer AB
   120. Love BC
   121. Mack ML
   122. Malpica N
   123. Marins T
   124. Maumet C
   125. McDonald K
   126. McGuire JT
   127. Melero H
   128. Méndez Leal AS
   129. Meyer B
   130. Meyer KN
   131. Mihai G
   132. Mitsis GD
   133. Moll J
   134. Nielson DM
   135. Nilsonne G
   136. Notter MP
   137. Olivetti E
   138. Onicas AI
   139. Papale P
   140. Patil KR
   141. Peelle JE
   142. Pérez A
   143. Pischedda D
   144. Poline JB
   145. Prystauka Y
   146. Ray S
   147. Reuter-Lorenz PA
   148. Reynolds RC
   149. Ricciardi E
   150. Rieck JR
   151. Rodriguez-Thompson AM
   152. Romyn A
   153. Salo T
   154. Samanez-Larkin GR
   155. Sanz-Morales E
   156. Schlichting ML
   157. Schultz DH
   158. Shen Q
   159. Sheridan MA
   160. Silvers JA
   161. Skagerlund K
   162. Smith A
   163. Smith DV
   164. Sokol-Hessner P
   165. Steinkamp SR
   166. Tashjian SM
   167. Thirion B
   168. Thorp JN
   169. Tinghög G
   170. Tisdall L
   171. Tompson SH
   172. Toro-Serey C
   173. Torre Tresols JJ
   174. Tozzi L
   175. Truong V
   176. Turella L
   177. van 't Veer AE
   178. Verguts T
   179. Vettel JM
   180. Vijayarajah S
   181. Vo K
   182. Wall MB
   183. Weeda WD
   184. Weis S
   185. White DJ
   186. Wisniewski D
   187. Xifra-Porxas A
   188. Yearling EA
   189. Yoon S
   190. Yuan R
   191. Yuen KSL
   192. Zhang L
   193. Zhang X
   194. Zosky JE
   195. Nichols TE
   196. Poldrack RA
   197. Schonberg T

   (2020) Variability in the analysis of a single neuroimaging dataset by many teams

   Nature 582:84–88.

   https://doi.org/10.1038/s41586-020-2314-9

   • PubMed
   • Google Scholar
9. 1. Brand A
   2. Allen L
   3. Altman M
   4. Hlava M
   5. Scott J

   (2015) Beyond authorship: attribution, contribution, collaboration, and credit

   Learned Publishing 28:151–155.

   https://doi.org/10.1087/20150211

   • Google Scholar
10. 1. Burgess CP
    2. Lak A
    3. Steinmetz NA
    4. Zatka-Haas P
    5. Bai Reddy C
    6. Jacobs EAK
    7. Linden JF
    8. Paton JJ
    9. Ranson A
    10. Schröder S
    11. Soares S
    12. Wells MJ
    13. Wool LE
    14. Harris KD
    15. Carandini M

    (2017) High-Yield methods for accurate Two-Alternative visual psychophysics in Head-Fixed mice

    Cell Reports 20:2513–2524.

    https://doi.org/10.1016/j.celrep.2017.08.047

    • PubMed
    • Google Scholar
11. 1. Busse L
    2. Ayaz A
    3. Dhruv NT
    4. Katzner S
    5. Saleem AB
    6. Schölvinck ML
    7. Zaharia AD
    8. Carandini M

    (2011) The detection of visual contrast in the behaving mouse

    Journal of Neuroscience 31:11351–11361.

    https://doi.org/10.1523/JNEUROSCI.6689-10.2011

    • PubMed
    • Google Scholar
12. 1. Button KS
    2. Ioannidis JP
    3. Mokrysz C
    4. Nosek BA
    5. Flint J
    6. Robinson ES
    7. Munafò MR

    (2013) Power failure: why small sample size undermines the reliability of neuroscience

    Nature Reviews Neuroscience 14:365–376.

    https://doi.org/10.1038/nrn3475

    • PubMed
    • Google Scholar
13. 1. Bycroft C
    2. Freeman C
    3. Petkova D
    4. Band G
    5. Elliott LT
    6. Sharp K
    7. Motyer A
    8. Vukcevic D
    9. Delaneau O
    10. O'Connell J
    11. Cortes A
    12. Welsh S
    13. Young A
    14. Effingham M
    15. McVean G
    16. Leslie S
    17. Allen N
    18. Donnelly P
    19. Marchini J

    (2018) The UK biobank resource with deep phenotyping and genomic data

    Nature 562:203–209.

    https://doi.org/10.1038/s41586-018-0579-z

    • PubMed
    • Google Scholar
14. 1. Camerer CF
    2. Dreber A
    3. Holzmeister F
    4. Ho TH
    5. Huber J
    6. Johannesson M
    7. Kirchler M
    8. Nave G
    9. Nosek BA
    10. Pfeiffer T
    11. Altmejd A
    12. Buttrick N
    13. Chan T
    14. Chen Y
    15. Forsell E
    16. Gampa A
    17. Heikensten E
    18. Hummer L
    19. Imai T
    20. Isaksson S
    21. Manfredi D
    22. Rose J
    23. Wagenmakers EJ
    24. Wu H

    (2018) Evaluating the replicability of social science experiments in nature and science between 2010 and 2015

    Nature Human Behaviour 2:637–644.

    https://doi.org/10.1038/s41562-018-0399-z

    • PubMed
    • Google Scholar
15. 1. Carandini M
    2. Churchland AK

    (2013) Probing perceptual decisions in rodents

    Nature Neuroscience 16:824–831.

    https://doi.org/10.1038/nn.3410

    • PubMed
    • Google Scholar
16. Report

    1. CERN Education, Communications and Outreach Group

    (2018)

    CERN Annual Report 2017

    CERN.

    • Google Scholar
17. 1. Charles AS
    2. Falk B
    3. Turner N
    4. Pereira TD
    5. Tward D
    6. Pedigo BD
    7. Chung J
    8. Burns R
    9. Ghosh SS
    10. Kebschull JM
    11. Silversmith W
    12. Vogelstein JT

    (2020) Toward Community-Driven big Open brain science: open big data and tools for structure, function, and genetics

    Annual Review of Neuroscience 43:441–464.

    https://doi.org/10.1146/annurev-neuro-100119-110036

    • PubMed
    • Google Scholar
18. 1. Chesler EJ
    2. Wilson SG
    3. Lariviere WR
    4. Rodriguez-Zas SL
    5. Mogil JS

    (2002) Influences of laboratory environment on behavior

    Nature Neuroscience 5:1101–1102.

    https://doi.org/10.1038/nn1102-1101

    • PubMed
    • Google Scholar
19. 1. Cohen MR
    2. Maunsell JH

    (2009) Attention improves performance primarily by reducing interneuronal correlations

    Nature Neuroscience 12:1594–1600.

    https://doi.org/10.1038/nn.2439

    • PubMed
    • Google Scholar
20. 1. Corrado GS
    2. Sugrue LP
    3. Seung HS
    4. Newsome WT

    (2005) Linear-Nonlinear-Poisson models of primate choice dynamics

    Journal of the Experimental Analysis of Behavior 84:581–617.

    https://doi.org/10.1901/jeab.2005.23-05

    • PubMed
    • Google Scholar
21. 1. Crabbe JC
    2. Wahlsten D
    3. Dudek BC

    (1999) Genetics of Mouse Behavior: Interactions with Laboratory Environment

    Science 284:1670–1672.

    https://doi.org/10.1126/science.284.5420.1670

    • Google Scholar
22. 1. de Lafuente V
    2. Romo R

    (2005) Neuronal correlates of subjective sensory experience

    Nature Neuroscience 8:1698–1703.

    https://doi.org/10.1038/nn1587

    • PubMed
    • Google Scholar
23. 1. de Vries SEJ
    2. Lecoq JA
    3. Buice MA
    4. Groblewski PA
    5. Ocker GK
    6. Oliver M
    7. Feng D
    8. Cain N
    9. Ledochowitsch P
    10. Millman D
    11. Roll K
    12. Garrett M
    13. Keenan T
    14. Kuan L
    15. Mihalas S
    16. Olsen S
    17. Thompson C
    18. Wakeman W
    19. Waters J
    20. Williams D
    21. Barber C
    22. Berbesque N
    23. Blanchard B
    24. Bowles N
    25. Caldejon SD
    26. Casal L
    27. Cho A
    28. Cross S
    29. Dang C
    30. Dolbeare T
    31. Edwards M
    32. Galbraith J
    33. Gaudreault N
    34. Gilbert TL
    35. Griffin F
    36. Hargrave P
    37. Howard R
    38. Huang L
    39. Jewell S
    40. Keller N
    41. Knoblich U
    42. Larkin JD
    43. Larsen R
    44. Lau C
    45. Lee E
    46. Lee F
    47. Leon A
    48. Li L
    49. Long F
    50. Luviano J
    51. Mace K
    52. Nguyen T
    53. Perkins J
    54. Robertson M
    55. Seid S
    56. Shea-Brown E
    57. Shi J
    58. Sjoquist N
    59. Slaughterbeck C
    60. Sullivan D
    61. Valenza R
    62. White C
    63. Williford A
    64. Witten DM
    65. Zhuang J
    66. Zeng H
    67. Farrell C
    68. Ng L
    69. Bernard A
    70. Phillips JW
    71. Reid RC
    72. Koch C

    (2020) A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex

    Nature Neuroscience 23:138–151.

    https://doi.org/10.1038/s41593-019-0550-9

    • PubMed
    • Google Scholar
24. 1. Dickinson ME
    2. Flenniken AM
    3. Ji X
    4. Teboul L
    5. Wong MD
    6. White JK
    7. Meehan TF
    8. Weninger WJ
    9. Westerberg H
    10. Adissu H
    11. Baker CN
    12. Bower L
    13. Brown JM
    14. Caddle LB
    15. Chiani F
    16. Clary D
    17. Cleak J
    18. Daly MJ
    19. Denegre JM
    20. Doe B
    21. Dolan ME
    22. Edie SM
    23. Fuchs H
    24. Gailus-Durner V
    25. Galli A
    26. Gambadoro A
    27. Gallegos J
    28. Guo S
    29. Horner NR
    30. Hsu CW
    31. Johnson SJ
    32. Kalaga S
    33. Keith LC
    34. Lanoue L
    35. Lawson TN
    36. Lek M
    37. Mark M
    38. Marschall S
    39. Mason J
    40. McElwee ML
    41. Newbigging S
    42. Nutter LM
    43. Peterson KA
    44. Ramirez-Solis R
    45. Rowland DJ
    46. Ryder E
    47. Samocha KE
    48. Seavitt JR
    49. Selloum M
    50. Szoke-Kovacs Z
    51. Tamura M
    52. Trainor AG
    53. Tudose I
    54. Wakana S
    55. Warren J
    56. Wendling O
    57. West DB
    58. Wong L
    59. Yoshiki A
    60. MacArthur DG
    61. Tocchini-Valentini GP
    62. Gao X
    63. Flicek P
    64. Bradley A
    65. Skarnes WC
    66. Justice MJ
    67. Parkinson HE
    68. Moore M
    69. Wells S
    70. Braun RE
    71. Svenson KL
    72. de Angelis MH
    73. Herault Y
    74. Mohun T
    75. Mallon AM
    76. Henkelman RM
    77. Brown SD
    78. Adams DJ
    79. Lloyd KC
    80. McKerlie C
    81. Beaudet AL
    82. Bućan M
    83. Murray SA
    84. International Mouse Phenotyping Consortium
    85. Jackson Laboratory
    86. Infrastructure Nationale PHENOMIN, Institut Clinique de la Souris (ICS)
    87. Charles River Laboratories
    88. MRC Harwell
    89. Toronto Centre for Phenogenomics
    90. Wellcome Trust Sanger Institute
    91. RIKEN BioResource Center

    (2016) High-throughput discovery of novel developmental phenotypes

    Nature 537:508–514.

    https://doi.org/10.1038/nature19356

    • PubMed
    • Google Scholar
25. 1. Ding L
    2. Gold JI

    (2013) The basal ganglia's contributions to perceptual decision making

    Neuron 79:640–649.

    https://doi.org/10.1016/j.neuron.2013.07.042

    • PubMed
    • Google Scholar
26. 1. Fan Y
    2. Gold JI
    3. Ding L

    (2018) Ongoing, rational calibration of reward-driven perceptual biases

    eLife 7:e36018.

    https://doi.org/10.7554/eLife.36018

    • PubMed
    • Google Scholar
27. 1. Feng S
    2. Holmes P
    3. Rorie A
    4. Newsome WT

    (2009) Can monkeys choose optimally when faced with noisy stimuli and unequal rewards?

    PLOS Computational Biology 5:e1000284.

    https://doi.org/10.1371/journal.pcbi.1000284

    • PubMed
    • Google Scholar
28. 1. Fish V
    2. Akiyama K
    3. Bouman K
    4. Chael A
    5. Johnson M
    6. Doeleman S
    7. Blackburn L
    8. Wardle J
    9. Freeman W
    10. the Event Horizon Telescope Collaboration

    (2016) Observing—and Imaging—Active Galactic Nuclei with the Event Horizon Telescope

    Galaxies 4:54.

    https://doi.org/10.3390/galaxies4040054

    • Google Scholar
29. Preprint

    1. Forscher PS
    2. Wagenmakers E-J
    3. Coles NA
    4. Silan MA
    5. IJzerman H

    (2020) A manifesto for team science

    PsyArXiv.

    https://doi.org/10.31234/osf.io/2mdxh

    • Google Scholar
30. 1. Frank MC
    2. Bergelson E
    3. Bergmann C
    4. Cristia A
    5. Floccia C
    6. Gervain J
    7. Hamlin JK
    8. Hannon EE
    9. Kline M
    10. Levelt C
    11. Lew-Williams C
    12. Nazzi T
    13. Panneton R
    14. Rabagliati H
    15. Soderstrom M
    16. Sullivan J
    17. Waxman S
    18. Yurovsky D

    (2017) A collaborative approach to infant research: promoting reproducibility, best practices, and Theory-Building

    Infancy 22:421–435.

    https://doi.org/10.1111/infa.12182

    • PubMed
    • Google Scholar
31. 1. Glickfeld LL
    2. Reid RC
    3. Andermann ML

    (2014) A mouse model of higher visual cortical function

    Current Opinion in Neurobiology 24:28–33.

    https://doi.org/10.1016/j.conb.2013.08.009

    • PubMed
    • Google Scholar
32. 1. Goodman SN
    2. Fanelli D
    3. Ioannidis JP

    (2016) What does research reproducibility mean?

    Science Translational Medicine 8:341ps12.

    https://doi.org/10.1126/scitranslmed.aaf5027

    • PubMed
    • Google Scholar
33. 1. Guo ZV
    2. Hires SA
    3. Li N
    4. O'Connor DH
    5. Komiyama T
    6. Ophir E
    7. Huber D
    8. Bonardi C
    9. Morandell K
    10. Gutnisky D
    11. Peron S
    12. Xu NL
    13. Cox J
    14. Svoboda K

    (2014) Procedures for behavioral experiments in head-fixed mice

    PLOS ONE 9:e88678.

    https://doi.org/10.1371/journal.pone.0088678

    • PubMed
    • Google Scholar
34. 1. Hanks TD
    2. Mazurek ME
    3. Kiani R
    4. Hopp E
    5. Shadlen MN

    (2011) Elapsed decision time affects the weighting of prior probability in a perceptual decision task

    Journal of Neuroscience 31:6339–6352.

    https://doi.org/10.1523/JNEUROSCI.5613-10.2011

    • PubMed
    • Google Scholar
35. 1. Harris CR
    2. Millman KJ
    3. van der Walt SJ
    4. Gommers R
    5. Virtanen P
    6. Cournapeau D
    7. Wieser E
    8. Taylor J
    9. Berg S
    10. Smith NJ
    11. Kern R
    12. Picus M
    13. Hoyer S
    14. van Kerkwijk MH
    15. Brett M
    16. Haldane A
    17. Del Río JF
    18. Wiebe M
    19. Peterson P
    20. Gérard-Marchant P
    21. Sheppard K
    22. Reddy T
    23. Weckesser W
    24. Abbasi H
    25. Gohlke C
    26. Oliphant TE

    (2020) Array programming with NumPy

    Nature 585:357–362.

    https://doi.org/10.1038/s41586-020-2649-2

    • PubMed
    • Google Scholar
36. 1. Herrnstein RJ

    (1961) Relative and absolute strength of response as a function of frequency of reinforcement

    Journal of the Experimental Analysis of Behavior 4:267–272.

    https://doi.org/10.1901/jeab.1961.4-267

    • PubMed
    • Google Scholar
37. 1. International Brain Laboratory

    (2017) An international laboratory for systems and computational neuroscience

    Neuron 96:1213–1218.

    https://doi.org/10.1016/j.neuron.2017.12.013

    • PubMed
    • Google Scholar
38. 1. Ioannidis JP

    (2005) Why most published research findings are false

    PLOS Medicine 2:e124.

    https://doi.org/10.1371/journal.pmed.0020124

    • PubMed
    • Google Scholar
39. 1. Kafkafi N
    2. Agassi J
    3. Chesler EJ
    4. Crabbe JC
    5. Crusio WE
    6. Eilam D
    7. Gerlai R
    8. Golani I
    9. Gomez-Marin A
    10. Heller R
    11. Iraqi F
    12. Jaljuli I
    13. Karp NA
    14. Morgan H
    15. Nicholson G
    16. Pfaff DW
    17. Richter SH
    18. Stark PB
    19. Stiedl O
    20. Stodden V
    21. Tarantino LM
    22. Tucci V
    23. Valdar W
    24. Williams RW
    25. Würbel H
    26. Benjamini Y

    (2018) Reproducibility and replicability of rodent phenotyping in preclinical studies

    Neuroscience & Biobehavioral Reviews 87:218–232.

    https://doi.org/10.1016/j.neubiorev.2018.01.003

    • PubMed
    • Google Scholar
40. 1. Koscielny G
    2. Yaikhom G
    3. Iyer V
    4. Meehan TF
    5. Morgan H
    6. Atienza-Herrero J
    7. Blake A
    8. Chen CK
    9. Easty R
    10. Di Fenza A
    11. Fiegel T
    12. Grifiths M
    13. Horne A
    14. Karp NA
    15. Kurbatova N
    16. Mason JC
    17. Matthews P
    18. Oakley DJ
    19. Qazi A
    20. Regnart J
    21. Retha A
    22. Santos LA
    23. Sneddon DJ
    24. Warren J
    25. Westerberg H
    26. Wilson RJ
    27. Melvin DG
    28. Smedley D
    29. Brown SD
    30. Flicek P
    31. Skarnes WC
    32. Mallon AM
    33. Parkinson H

    (2014) The international mouse phenotyping consortium web portal, a unified point of access for knockout mice and related phenotyping data

    Nucleic Acids Research 42:D802–D809.

    https://doi.org/10.1093/nar/gkt977

    • PubMed
    • Google Scholar
41. 1. Lak A
    2. Hueske E
    3. Hirokawa J
    4. Masset P
    5. Ott T
    6. Urai AE
    7. Donner TH
    8. Carandini M
    9. Tonegawa S
    10. Uchida N
    11. Kepecs A

    (2020a) Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon

    eLife 9:e49834.

    https://doi.org/10.7554/eLife.49834

    • PubMed
    • Google Scholar
42. 1. Lak A
    2. Okun M
    3. Moss MM
    4. Gurnani H
    5. Farrell K
    6. Wells MJ
    7. Reddy CB
    8. Kepecs A
    9. Harris KD
    10. Carandini M

    (2020b) Dopaminergic and prefrontal basis of learning from sensory confidence and reward value

    Neuron 105:700–711.

    https://doi.org/10.1016/j.neuron.2019.11.018

    • PubMed
    • Google Scholar
43. 1. Lau B
    2. Glimcher PW

    (2005) Dynamic response-by-response models of matching behavior in rhesus monkeys

    Journal of the Experimental Analysis of Behavior 84:555–579.

    https://doi.org/10.1901/jeab.2005.110-04

    • PubMed
    • Google Scholar
44. 1. Liston DB
    2. Stone LS

    (2008) Effects of prior information and reward on oculomotor and perceptual choices

    Journal of Neuroscience 28:13866–13875.

    https://doi.org/10.1523/JNEUROSCI.3120-08.2008

    • PubMed
    • Google Scholar
45. 1. Lopes G
    2. Bonacchi N
    3. Frazão J
    4. Neto JP
    5. Atallah BV
    6. Soares S
    7. Moreira LuÃs
    8. Matias S
    9. Itskov PM
    10. Correia P
    11. Medina RE
    12. Calcaterra L
    13. Dreosti E
    14. Paton JJ
    15. Kampff AR
    16. Lopes G
    17. Bonacchi N
    18. Frazão J
    19. Moreira L
    20. Correia PA

    (2015) Bonsai: an event-based framework for processing and controlling data streams

    Frontiers in Neuroinformatics 9:91.

    https://doi.org/10.1101/006791

    • Google Scholar
46. 1. Lopes G
    2. Farrell K
    3. Horrocks EA
    4. Lee CY
    5. Morimoto MM
    6. Muzzu T
    7. Papanikolaou A
    8. Rodrigues FR
    9. Wheatcroft T
    10. Zucca S
    11. Solomon SG
    12. Saleem AB

    (2021) Creating and controlling visual environments using BonVision

    eLife 10:e65541.

    https://doi.org/10.7554/eLife.65541

    • PubMed
    • Google Scholar
47. 1. Makel MC
    2. Plucker JA
    3. Hegarty B

    (2012) Replications in psychology research: how often do they really occur?

    Psychol. Sci. J. Assoc. Psychol. Sci 7:537–542.

    https://doi.org/10.1177/1745691612460688

    • Google Scholar
48. 1. Mathis A
    2. Mamidanna P
    3. Cury KM
    4. Abe T
    5. Murthy VN
    6. Mathis MW
    7. Bethge M

    (2018) DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

    Nature Neuroscience 21:1281–1289.

    https://doi.org/10.1038/s41593-018-0209-y

    • PubMed
    • Google Scholar
49. 1. McGinley MJ
    2. Vinck M
    3. Reimer J
    4. Batista-Brito R
    5. Zagha E
    6. Cadwell CR
    7. Tolias AS
    8. Cardin JA
    9. McCormick DA

    (2015) Waking state: rapid variations modulate neural and behavioral responses

    Neuron 87:1143–1161.

    https://doi.org/10.1016/j.neuron.2015.09.012

    • PubMed
    • Google Scholar
50. Preprint

    1. Mendonça AG
    2. Drugowitsch J
    3. Vicente MI
    4. DeWitt E
    5. Pouget A
    6. Mainen ZF

    (2018) The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs

    bioRxiv.

    https://doi.org/10.1101/501858

    • Google Scholar
51. Preprint

    1. Miller KJ
    2. Botvinick MM
    3. Brody CD

    (2019) From predictive models to cognitive models: an analysis of rat behavior in the two-armed bandit task

    bioRxiv.

    https://doi.org/10.1101/461129

    • Google Scholar
52. 1. Newsome WT
    2. Britten KH
    3. Movshon JA

    (1989) Neuronal correlates of a perceptual decision

    Nature 341:52–54.

    https://doi.org/10.1038/341052a0

    • PubMed
    • Google Scholar
53. 1. Norton EH
    2. Acerbi L
    3. Ma WJ
    4. Landy MS

    (2019) Human online adaptation to changes in prior probability

    PLOS Computational Biology 15:e1006681.

    https://doi.org/10.1371/journal.pcbi.1006681

    • PubMed
    • Google Scholar
54. 1. O'Connor DH
    2. Huber D
    3. Svoboda K

    (2009) Reverse engineering the mouse brain

    Nature 461:923–929.

    https://doi.org/10.1038/nature08539

    • PubMed
    • Google Scholar
55. 1. Pedregosa F
    2. Varoquaux G
    3. Gramfort A
    4. Michel V
    5. Thirion B
    6. Grisel O
    7. Blondel M
    8. Prettenhofer P
    9. Weiss R
    10. Dubourg V

    (2011)

    Scikit-learn: machine learning in Python

    Journal of Machine Learning Research : JMLR 12:2825–2830.

    • Google Scholar
56. 1. Pinto L
    2. Koay SA
    3. Engelhard B
    4. Yoon AM
    5. Deverett B
    6. Thiberge SY
    7. Witten IB
    8. Tank DW
    9. Brody CD

    (2018) An Accumulation-of-Evidence task using visual pulses for mice navigating in virtual reality

    Frontiers in Behavioral Neuroscience 12:36.

    https://doi.org/10.3389/fnbeh.2018.00036

    • PubMed
    • Google Scholar
57. 1. Pisupati S
    2. Chartarifsky-Lynn L
    3. Khanal A
    4. Churchland AK

    (2021) Lapses in perceptual decisions reflect exploration

    eLife 10:e55490.

    https://doi.org/10.7554/eLife.55490

    • PubMed
    • Google Scholar
58. 1. Poddar R
    2. Kawai R
    3. Ölveczky BP

    (2013) A fully automated high-throughput training system for rodents

    PLOS ONE 8:e83171.

    https://doi.org/10.1371/journal.pone.0083171

    • PubMed
    • Google Scholar
59. 1. Poldrack RA
    2. Gorgolewski KJ

    (2014) Making big data open: data sharing in neuroimaging

    Nature Neuroscience 17:1510–1517.

    https://doi.org/10.1038/nn.3818

    • PubMed
    • Google Scholar
60. Software

    1. Reback J
    2. McKinney W
    3. jbrockmendel B
    4. den JV
    5. Augspurger T
    6. Cloud P
    7. gfyoung S
    8. Klein A
    9. Roeschke M

    (2020) pandas-dev/pandas, version Pandas 1.0.1

    Zenodo.

    https://doi.org/10.5281/zenodo.3644238
61. 1. Romo R
    2. Lemus L
    3. de Lafuente V

    (2012) Sense, memory, and decision-making in the somatosensory cortical network

    Current Opinion in Neurobiology 22:914–919.

    https://doi.org/10.1016/j.conb.2012.08.002

    • PubMed
    • Google Scholar
62. 1. Roy NA
    2. Bak JH
    3. Akrami A
    4. Brody CD
    5. Pillow JW
    6. International Brain Laboratory

    (2021) Extracting the dynamics of behavior in sensory decision-making experiments

    Neuron 109:597–610.

    https://doi.org/10.1016/j.neuron.2020.12.004

    • PubMed
    • Google Scholar
63. 1. Scott BB
    2. Constantinople CM
    3. Erlich JC
    4. Tank DW
    5. Brody CD

    (2015) Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats

    eLife 4:e11308.

    https://doi.org/10.7554/eLife.11308

    • PubMed
    • Google Scholar
64. Conference

    1. Seabold S
    2. Perktold J

    (2010)

    Statsmodels: econometric and statistical modeling with Python

    Proceedings of the 9th Python in Science Conference.

    • Google Scholar
65. 1. Seabrook TA
    2. Burbridge TJ
    3. Crair MC
    4. Huberman AD

    (2017) Architecture, function, and assembly of the mouse visual system

    Annual Review of Neuroscience 40:499–538.

    https://doi.org/10.1146/annurev-neuro-071714-033842

    • PubMed
    • Google Scholar
66. 1. Shadlen MN
    2. Kiani R

    (2013) Decision making as a window on cognition

    Neuron 80:791–806.

    https://doi.org/10.1016/j.neuron.2013.10.047

    • PubMed
    • Google Scholar
67. Software

    1. Smith NJ
    2. Hudon C
    3. broessli SS
    4. Quackenbush P
    5. Hudson-Doyle M
    6. Humber M
    7. Leinweber K
    8. Kibirige H
    9. Davidson-Pilon C

    (2018) pydata/patsy, version v0.5.1

    Zenodo.

    https://doi.org/10.5281/zenodo.1472929
68. 1. Sorge RE
    2. Martin LJ
    3. Isbester KA
    4. Sotocinal SG
    5. Rosen S
    6. Tuttle AH
    7. Wieskopf JS
    8. Acland EL
    9. Dokova A
    10. Kadoura B
    11. Leger P
    12. Mapplebeck JC
    13. McPhail M
    14. Delaney A
    15. Wigerblad G
    16. Schumann AP
    17. Quinn T
    18. Frasnelli J
    19. Svensson CI
    20. Sternberg WF
    21. Mogil JS

    (2014) Olfactory exposure to males, including men, causes stress and related analgesia in rodents

    Nature Methods 11:629–632.

    https://doi.org/10.1038/nmeth.2935

    • PubMed
    • Google Scholar
69. 1. Tanner WP
    2. Swets JA

    (1954) A decision-making theory of visual detection

    Psychological Review 61:401–409.

    https://doi.org/10.1037/h0058700

    • PubMed
    • Google Scholar
70. 1. Terman M
    2. Terman JS

    (1972) Concurrent variation of response Bias and sensitivity in an operant-psychophysical test

    Perception & Psychophysics 11:428–432.

    https://doi.org/10.3758/BF03206285

    • Google Scholar
71. Report

    1. The International Brain Laboratory

    (2020a) Behavior: Appendix 1: IBL Protocol for Headbar Implant Surgery in Mice

    figshare.

    https://doi.org/10.6084/m9.figshare.11634726.v4

    • Google Scholar
72. Report

    1. The International Brain Laboratory

    (2020b) Behavior: Appendix 2: IBL Protocol for Mice Training

    figshare.

    https://doi.org/10.6084/m9.figshare.11634729.v3

    • Google Scholar
73. Report

    1. The International Brain Laboratory

    (2021a) Behavior: Appendix 3: IBL Protocol for Setting Up the Behavioral Training Rig

    figshare.

    https://doi.org/10.6084/m9.figshare.11634732.v6

    • Google Scholar
74. Software

    1. The International Brain Laboratory

    (2021b) paper-behavior, version swh:1:rev:edc453189104a1f76f4b2ab230cd86f2140e3f63

    Software Heritage.

    https://archive.softwareheritage.org/swh:1:rev:edc453189104a1f76f4b2ab230cd86f2140e3f63
75. 1. Tuttle AH
    2. Philip VM
    3. Chesler EJ
    4. Mogil JS

    (2018) Comparing phenotypic variation between inbred and outbred mice

    Nature Methods 15:994–996.

    https://doi.org/10.1038/s41592-018-0224-7

    • PubMed
    • Google Scholar
76. 1. Urai AE
    2. Braun A
    3. Donner TH

    (2017) Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias

    Nature Communications 8:14637.

    https://doi.org/10.1038/ncomms14637

    • Google Scholar
77. 1. Urai AE
    2. Aguillon-Rodriguez V
    3. Laranjeira IC
    4. Cazettes F
    5. Mainen ZF
    6. Churchland AK
    7. International Brain Laboratory

    (2021) Citric acid water as an alternative to water restriction for High-Yield mouse behavior

    Eneuro 8:ENEURO.0230-20.2020.

    https://doi.org/10.1523/ENEURO.0230-20.2020

    • PubMed
    • Google Scholar
78. 1. Voelkl B
    2. Altman NS
    3. Forsman A
    4. Forstmeier W
    5. Gurevitch J
    6. Jaric I
    7. Karp NA
    8. Kas MJ
    9. Schielzeth H
    10. Van de Casteele T
    11. Würbel H

    (2020) Reproducibility of animal research in light of biological variation

    Nature Reviews Neuroscience 21:384–393.

    https://doi.org/10.1038/s41583-020-0313-3

    • PubMed
    • Google Scholar
79. 1. Waskom M

    (2021) Seaborn: statistical data visualization

    Journal of Open Source Software 6:3021.

    https://doi.org/10.21105/joss.03021

    • Google Scholar
80. 1. Whiteley L
    2. Sahani M

    (2008) Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes

    Journal of Vision 8:2.

    https://doi.org/10.1167/8.3.2

    • PubMed
    • Google Scholar
81. 1. Wool LE
    2. International Brain Laboratory

    (2020) Knowledge across networks: how to build a global neuroscience collaboration

    Current Opinion in Neurobiology 65:100–107.

    https://doi.org/10.1016/j.conb.2020.10.020

    • PubMed
    • Google Scholar
82. Preprint

    1. Yatsenko D
    2. Walker EY
    3. Tolias A

    (2018) DataJoint: a simpler relational data model

    arXiv.

    https://arxiv.org/abs/1807.11104

    • Google Scholar

Author details

1. The International Brain Laboratory

   For correspondence

   info+behavior@internationalbrainlab.org

2. Valeria Aguillon-Rodriguez

   Cold Spring Harbor Laboratory, New York, United States

   Contribution

   Methodology: built, designed and tested rig assembly (equal); developed protocols for surgery, husbandry and animal training (equal); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal)

   Competing interests

   No competing interests declared

3. Dora Angelaki

   Center for Neural Science, New York University, New York, United States

   Contribution

   Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding Acquisition: acquired funding (support)

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-9650-8962

4. Hannah Bayer

   Zuckerman Institute, Columbia University, New York, United States

   Contribution

   Writing - Original draft: wrote and curated the appendix protocols (support); Writing - Review and editing: edited the paper (lead); Supervision: managed and coordinated team (support)

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5644-4124

5. Niccolo Bonacchi

   Champalimaud Centre for the Unknown, Lisbon, Portugal

   Contribution

   Methodology: built, designed and tested rig assembly (lead); designed and delivered rig components (support); piloted candidate behavioral tasks (equal); developed final behavioral task (equal); developed protocols for surgery, husbandry and animal training (support); standardized licenses and experimental protocols across institutions (equal); Software: developed data acquisition software and infrastructure (lead); Validation: maintained and validated analysis code (support); Formal analysis: analyzed data (support); Invetigation: built and maintained rigs, performed surgeries, collected behavioral data (support); Data curation: curated data and metadata (equal); Writing - Original draft: wrote the first version of the paper (support); wrote and curated the appendix protocols (support); Writing - Review and editing: edited the paper (support); Supervision: managed and coordinated team (support); Project administration: managed and coordinated research outputs (support); Funding acquisition: acquired funding (support)

   Competing interests

   No competing interests declared

6. Matteo Carandini

   UCL Institute of Ophthalmology, University College London, London, United Kingdom

   Contribution

   Conceptualization: defined composition and scope of the paper (lead); Resources: hosted the research (equal); Writing - Original draft: wrote the first version of the paper (equal); wrote the second version of the paper (lead); Writing - Review and editing: edited the paper (equal); Supervision: supervised local laboratory research (equal); managed and coordinated team (lead); Project administration: managed and coordinated research outputs (lead); Funding acquisition: acquired funding (equal)

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4880-7682

7. Fanny Cazettes

   Champalimaud Centre for the Unknown, Lisbon, Portugal

   Contribution

   Methodology: piloted candidate behavioral tasks (lead); developed final behavioral task (support); developed protocols for surgery, husbandry and animal training (support); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote the first version of the paper (support); Supervision: managed and coordinated team (support)

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-9648-4761

8. Gaelle Chapuis

   Wolfson Institute for Biomedical Research, University College London, London, United Kingdom

   Contribution

   Conceptualization: defined composition and scope of the paper (equal); Methodology: developed protocols for surgery, husbandry and animal training (lead); designed and delivered rig components (support); standardized licenses and experimental protocols across institutions (lead); Validation: maintained and validated analysis code (support); Formal analysis: analyzed data (support); Data curation: curated data and metadata (support); Writing - Original draft: wrote the second version of the paper (equal); wrote and curated the appendix protocols (lead); Writing - Review and editing: edited the paper (support); Visualization: designed and created figures (support); Supervision: managed and coordinated team (lead); Project administration: managed and coordinated research outputs (lead); Funding acqusition: acquired funding (support)

   Competing interests

   No competing interests declared

9. Anne K Churchland

   Cold Spring Harbor Laboratory, New York, United States

   Present address

   David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, United States

   Contribution

   Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding acquistion: acquired funding (lead)

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-3205-3794

10. Yang Dan

    Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, United States

    Contribution

    Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-3818-877X

11. Eric Dewitt

    Champalimaud Centre for the Unknown, Lisbon, Portugal

    Contribution

    Conceptualization: defined composition and scope of the paper (support); Methodology: developed final behavioral task (equal); piloted candidate behavioral tasks (equal); developed protocols for surgery, husbandry and animal training (support); Writing - Review and editing: edited the paper (support); Supervision: managed and coordinated team (support); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

12. Mayo Faulkner

    Wolfson Institute for Biomedical Research, University College London, London, United Kingdom

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote and curated the appendix protocols (equal)

    Competing interests

    No competing interests declared

13. Hamish Forrest

    UCL Institute of Ophthalmology, University College London, London, United Kingdom

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (support)

    Competing interests

    No competing interests declared

14. Laura Haetzel

    Princeton Neuroscience Institute, Princeton University, Princeton, United States

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal)

    Competing interests

    No competing interests declared

15. Michael Häusser

    Wolfson Institute for Biomedical Research, University College London, London, United Kingdom

    Contribution

    Resources: hosted the research (equal); Writing - Review and editing: edited the paper (support); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (lead)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-2673-8957

16. Sonja B Hofer

    Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom

    Contribution

    Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

17. Fei Hu

    Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, United States

    Contribution

    Methodology: standardized licenses and experimental protocols across institutions (equal); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0001-7827-9548

18. Anup Khanal

    Cold Spring Harbor Laboratory, New York, United States

    Present address

    David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, United States

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-8929-7984

19. Christopher Krasniak

    1. Cold Spring Harbor Laboratory, New York, United States
    2. Watson School of Biological Sciences, New York, United States

    Contribution

    Methodology: piloted candidate behavioral tasks (equal); developed protocols for surgery, husbandry and animal training (equal); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote the first version of the paper (equal); wrote and curated the appendix protocols (support); Writing - Review and editing: edited the paper (support)

    Competing interests

    No competing interests declared

20. Ines Laranjeira

    Champalimaud Centre for the Unknown, Lisbon, Portugal

    Contribution

    Methodology: piloted candidate behavioral tasks (equal); developed protocols for surgery, husbandry and animal training (equal); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote and curated the appendix protocols (support)

    Competing interests

    No competing interests declared

21. Zachary F Mainen

    Champalimaud Centre for the Unknown, Lisbon, Portugal

    Contribution

    Formal analysis: analyzed data (support); Resources: hosted the research (equal); Writing - Review and editing: edited the paper (equal); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (lead)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0001-7913-9109

22. Guido Meijer

    Champalimaud Centre for the Unknown, Lisbon, Portugal

    Contribution

    Conceptualization: defined composition and scope of the paper (equal); Methodology: developed final behavioral task (equal); built, designed and tested rig assembly (lead); standardized licenses and experimental protocols across institutions (equal); Validation: maintained and validated analysis code (equal); Formal analysis: analyzed data (lead); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote the second version of the paper (equal); wrote and curated the appendix protocols (equal); Visualization: designed and created figures (lead)

    Competing interests

    No competing interests declared

23. Nathaniel J Miska

    Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Review and editing: edited the paper (support)

    Competing interests

    No competing interests declared

24. Thomas D Mrsic-Flogel

    Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom

    Contribution

    Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

25. Masayoshi Murakami

    Champalimaud Centre for the Unknown, Lisbon, Portugal

    Present address

    Department of Neurophysiology, University of Yamanashi, Kōfu, Japan

    Contribution

    Methodology: built, designed and tested rig assembly (support); piloted candidate behavioral tasks (support)

    Competing interests

    No competing interests declared

26. Jean-Paul Noel

    Center for Neural Science, New York University, New York, United States

    Contribution

    Methodology: standardized licenses and experimental protocols across institutions (equal); Formal analysis: analyzed data (support); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote the first version of the paper (lead); Writing - Review and editing: edited the paper (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0001-5297-3363

27. Alejandro Pan-Vazquez

    Princeton Neuroscience Institute, Princeton University, Princeton, United States

    Contribution

    Conceptualization: defined composition and scope of the paper (equal); Methodology: standardized licenses and experimental protocols across institutions (equal); developed protocols for surgery, husbandry and animal training (support); Validation: maintained and validated analysis code (support); Formal analysis: analyzed data (lead); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Original draft: wrote the second version of the paper (equal); Writing - Review and editing: edited the paper (equal); Visualization: designed and created figures (lead)

    Competing interests

    No competing interests declared

28. Cyrille Rossant

    UCL Queen Square Institute of Neurology, University College London, London, United Kingdom

    Contribution

    Data curation: curated data and metadata (support)

    Competing interests

    No competing interests declared

29. Joshua Sanders

    Sanworks LLC, New York, United States

    Contribution

    Methodology: designed and delivered rig components (lead); built, designed and tested rig assembly (equal)

    Competing interests

    JIS is the owner of Sanworks LLC which provides hardware and consulting for the experimental set-up described in this work.

30. Karolina Socha

    UCL Institute of Ophthalmology, University College London, London, United Kingdom

    Contribution

    Methodology: developed protocols for surgery, husbandry and animal training (equal); built, designed and tested rig assembly (support); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Writing - Review and editing: edited the paper (support)

    Competing interests

    No competing interests declared

31. Rebecca Terry

    UCL Queen Square Institute of Neurology, University College London, London, United Kingdom

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (support)

    Competing interests

    No competing interests declared

32. Anne E Urai

    1. Cold Spring Harbor Laboratory, New York, United States
    2. Cognitive Psychology Unit, Leiden University, Leiden, Netherlands

    Contribution

    Conceptualization: defined composition and scope of the paper (equal); Methodology: built, designed and tested rig assembly (support); piloted candidate behavioral tasks (equal); developed final behavioral task (equal); developed protocols for surgery, husbandry and animal training (equal); Validation: maintained and validated analysis code (equal); Formal analysis: analyzed data (lead); Investigation: built and maintained rigs, performed surgeries, collected behavioral data (equal); Data curation: curated data and metadata (support); Writing - Original draft: wrote the second version of the paper (equal); wrote and curated the appendix protocols (support); Visualization: designed and created figures (lead); created data visualizations (lead); Supervision: managed and coordinated team (support); Project administration: managed and coordinated research outputs (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0001-5270-6513

33. Hernando Vergara

    Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (support); Writing - Original draft: wrote and curated the appendix protocols (support)

    Competing interests

    No competing interests declared

34. Miles Wells

    UCL Queen Square Institute of Neurology, University College London, London, United Kingdom

    Contribution

    Conceptualization: defined composition and scope of the paper (equal); Methodology: built, designed and tested rig assembly (support); piloted candidate behavioral tasks (lead); developed final behavioral task (equal); Validation: maintained and validated analysis code (equal); Formal analysis: analyzed data (equal); Data curation: curated data and metadata (lead); Writing - Original draft: wrote the second version of the paper (equal); Visualization: designed and created figures (support); created data visualizations (support)

    Competing interests

    No competing interests declared

35. Christian J Wilson

    Center for Neural Science, New York University, New York, United States

    Contribution

    Investigation: built and maintained rigs, performed surgeries, collected behavioral data (support)

    Competing interests

    No competing interests declared

36. Ilana B Witten

    Princeton Neuroscience Institute, Princeton University, Princeton, United States

    Contribution

    Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0003-0548-2160

37. Lauren E Wool

    UCL Queen Square Institute of Neurology, University College London, London, United Kingdom

    Contribution

    Methodology: built, designed and tested rig assembly (lead); developed final behavioral task (equal); developed protocols for surgery, husbandry and animal training (support); Writing - Original draft: wrote the first version of the paper (lead); Writing - Review and editing: edited the paper (equal); Funding acquisition: acquired funding (support)

    Competing interests

    No competing interests declared

38. Anthony M Zador

    Cold Spring Harbor Laboratory, New York, United States

    Contribution

    Resources: hosted the research (equal); Supervision: supervised local laboratory research (equal); managed and coordinated team (equal); Funding acquisition: acquired funding (equal)

    Competing interests

    No competing interests declared

• 15,701

  views

• 1,633

  downloads

• 108

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.