We developed a novel experimental approach, the ‘BD apricot task’, to study many-alternative decisions under uncertainty and limited resources by confronting participants with a BD dilemma. Participants played a gamified version of the task in which they were presented with several virtual apricot suppliers (10 or 32) having different, and unknown, probabilities of good-quality apricots (Materials and methods; see Video 1). These probabilities were independently generated for each supplier in each trial from a beta prior distribution. At the beginning of each trial, participants received a budget of a few coins, which defined their sampling capacity in that trial. Each coin was used to buy one apricot from a selected supplier (Figure 1a). Once all coins had been spent (Figure 1b), participants discovered which of the sampled apricots were of good quality, and which ones were bad (Figure 1c), whose outcomes followed a Bernoulli process with the unknown probabilities of good-quality apricots for each supplier. These probabilities were sampled independently for each supplier and trial from a beta distribution. Based on the observed outcomes, participants could estimate the probabilities from the sampled suppliers, and make a final purchase of 100 apricots from the one they considered to be the best (Figure 1d). Only one of the sampled alternatives could be selected for the final purchase. Their goal was to maximize the number of good-quality apricots collected throughout the experiment through the implementation of an informative sampling strategy, adapted both to the sampling capacity and to the environment richness. The task was intuitive and easily grasped by most participants. No instructions about the underlying probability generative model was provided as the context was informative enough to aid task understanding (Schustek et al., 2019).

Figure 1

Download asset Open asset

Video 1

Download asset

Two different ranges of sampling capacity, narrow (2–10 samples per trial) and wide (2–32), were tested to zoom-in relevant behavioural regimes (see Materials and methods and Table 1). For each range, we used three environments differing in the parameters of the beta distribution used to generate probabilities of good-quality apricots for each supplier. By increasing the average probability of good alternatives, we can increase the richness of the environment to move from a ‘poor’, to a ‘neutral’, up to a ‘rich’ environment (beta prior means: poor, 0.25; neutral, 0.50; and rich, 0.75). While environments and capacity ranges were run in a block fashion, capacity in each was chosen randomly in each trial from the underlying range in that block. Environments were tested within and between subjects in two different sets of experiments, to address potential learning effects.

We observed in all environments that participants’ sampling behaviour of suppliers follows a pure-breadth strategy at low capacity (Figure 2A–B), whereby they sample close to as many suppliers as coins they have. Indeed, at low capacity ($C\sim \mathrm{2,3}$) the number $M$ of sampled suppliers averaged over trials and participants is close to $C$ (Figure 2A) and the ratio $M/C$ reaches very close to 1 (Figure 2B). At higher capacities, sampling progressively evolves towards a trade-off between breadth and depth. Therefore, although participants could have sampled more suppliers to learn about, they preferred to focus sampling capacity on a rather small fraction of suppliers, as shown by the decline of the ratio $M/C$ towards values of around 0.4 at the highest capacities tested (Figure 2B; right panels). Although these are predicted features and thus signs of optimal behaviour, we did not observe a fast transition between the low- and high-capacity regimes, as previously predicted (Moreno-Bote et al., 2020, see also Appendix 1—figure 1A). This could simply result from averaging over participants having different transition points, or by having allocation noise, which would smooth out fast transitions. As shown below, models with allocation noise account for the smoothness of this transitions and are better models in predicting the behaviour than the noise-free optimal model.

Figure 2

Download asset Open asset

The effect of environmental richness was also clearly visible in our data, with richer environments typically causing a stronger preference for depth sampling (Figure 2A–B; different colours), consistent with predictions. Therefore, as it becomes easier for the participant to find good options, they prefer to neglect an even larger number of suppliers and focus capacity to examine fewer ones. We also observed systematic deviations from optimality (Figure 2A; black lines). In particular, despite participants’ strategy was overall strongly dependent on environment richness, as predicted, this was mostly due to a strong effect of the poor environment (Figure 2A–B; red lines), whereas they were not sensitive to the difference between neutral and rich environments (green and blue lines).

To test quantitatively the effect of environmental richness on the BD trade-offs, we fitted participant’s individual data in each environment using three models. These models were chosen based both on our predictions and on previous observations. First, a piece-wise power-law model (W), having a fast transition at some arbitrary capacity value, was selected as it is the one anticipated by the ideal observer. Second, as previously said, we visually noticed that for a majority of participants the transition between pure-breadth and BD trade-off was gradual (see Figure 2), so we decided to capture their sampling strategy using two simpler models: a linear model (L) and a power-law model (P). Indeed, it has been predicted that once the BD trade-off established, the number of alternatives sampled $M$ approximately increases with a power-law behaviour (Moreno-Bote et al., 2020). We observed that the power-law model (${\displaystyle R_{adj}^2=0.96\pm 0.04}$ mean ± s.d.) showed significantly better fits than the linear model (${\displaystyle R_{adj}^2=0.92\pm 0.06}$ mean ± s.d.${\displaystyle ;V=404,p<2.2\times {10}^{-16}}$). Moreover, the linear piece-wise model was not significantly better than the power-law model for all participants in all environments individually (ANOVAs, with ${\displaystyle \alpha =.05}$).

We compared the exponent estimated from the power-law fits in each environment and experimental design (Figure 3) using within- (W10 and 32) or between-subjects (B10 and 32) designs. The results demonstrated a significant effect of the environment on the exponent in all designs (one-way ANOVAs, B10: $F_1=13.17,p=7.51\times {10}^{-4}$ ; W10: $F_1=31.05,p=3.37\times {10}^{-5}$ ; W32: ${\displaystyle F_1=15.12,p=.0012}$), except for the between-subjects design and the larger range of capacities tested (B32: $F_1=0.82,p=.37$). Overall, effect sizes were larger in designs with narrow (B10: ${\omega }_G^2=.213$, W10: ${\omega }_G^2=.121$) compared to wide ranges of capacities (B32: ${\omega }_G^2=-.004$, W32: ${\omega }_G^2=.072$) and the non-result observed may stem from an insufficient sample size (achieved power in B32 is 14.7%, against 97.8% in W32, 95.2% in B10 and 94.7% in W10). As suggested by visual inspection above, there is no significant difference in the exponents between the neutral (median values, W10: $m_{neutral}=0.72$, B10: $m_{neutral}=0.68$, W32: $m_{neutral}=0.70$) and rich (W10: $m_{rich}=0.71$, B10: $m_{rich}=0.66$, W32: $m_{rich}=0.66$) environments in any of the designs (t-tests with Bonferroni correction, W10: $t_{17}=0.57,p_{adj}=1$, B10: $t_{23.9}=0.24,p_{adj}=1$, W32: $t_{17}=2.04,p_{adj}=.17$), although strong and significant differences were typically observed between rich and poor (W10: $m_{poor}=0.82$, B10: $m_{poor}=0.84$, W32: $m_{poor}=0.75$) environments (W10: $t_{17}=5.57,p_{adj}=1.01\times {10}^{-4}$ , B10: $t_{20.8}=3.51,p_{adj}=.006$, W32: $t_{17}=3.89,p_{adj}=.004$). Significant differences were also observed between poor and neutral environments in the designs with smaller capacities (W10: $t_{17}=3.25,p_{adj}=.014$ , B10: $t_{26.7}=4.45,p_{adj}=4.11\times {10}^{-4}$). Overall, we observed in the three designs mentioned a decrease of the exponent as the environment gets richer. This suggests an adaptation of participants’ sampling behaviour to the environment, with sampling depth increasing with the richness of environment.

Figure 3

Download asset Open asset

We have demonstrated that, as expected, participants’ sampling strategy is modulated by the environment richness. We now investigate whether the strategy used coincides with the optimal sampling behaviour or if some deviations are observable. Pooling the data of the four experimental designs together, we confirmed our previous results and found a significant effect overall of the environment on the exponent of the power law (ANOVA, $F_1=21.03,p=8.23\times {10}^{-6}$). Importantly, there was no significant effect of the experimental design (ANOVA, $F_3=0.76,p=.52$) nor a significant interaction between the environment and the experimental design (ANOVA, $F_3=1.40,p=.24$), thus confirming that the effect of the environment can be studied on the whole dataset independently of the experimental design. In order to quantify deviations of participants’ sampling strategies from the optimal strategy, we fitted the optimal values of the number $M$ of sampled suppliers for all capacities (${\displaystyle C=\left\{2-10,16,32\right\}}$) using the power-law model previously described (see Materials and methods) and extracted the power-law exponent. Comparing the observed values of the exponent to the optimal ones (see Table 2), we observed that participants’ sampling strategy significantly shifted towards excessive depth in the poor environment (t-test, $t_{65}=-5.66,p_{adj}=3.72{\times 10}^{-7}$), and a similar tendency occurred in the neutral environment (t-test, $t_{65}=-1.74,p_{adj}=.087$). In contrast, participants’ sampling strategy deviated significantly from optimality in the direction of excessive breadth in the rich environment (t-test, $t_{65}=4.06,p_{adj}=1.36{\times 10}^{-4}$).

Looking in greater detail at which capacities these deviations occur, we computed the differences between the optimal number $M_{opt}$ of sampled suppliers and the observed $M$ for each capacity and environment and observed again a significant effect of the environment on the differences (Scheirer-Ray-Hare test, ${\displaystyle H_2=159.67,p<2.2\times {10}^{-16}}$), but also a significant effect of capacity (${\displaystyle H_{10}=98.07,p=2.2\times {10}^{-16}}$) and a significant interaction between the environment and capacity ($H_{10}=84.08,p=7.89{\times 10}^{-10}$), showing that the participants' bias towards excessive depth or breadth varies with the environment and the capacity. In particular, we observed that at low capacity, the difference between $M_{opt}$ and the observed $M$ tended to be positive, which is especially visible in the poor and neutral environments (Appendix 1—figure 1 and exhaustive analyses presented in Supplementary file 1). In contrast, at high capacity this difference tends to be negative, which is especially noticeable in the neutral and rich environments.

The results of these analyses indicate some deviations from optimality. Participants have a bias to sample more deeply than optimal at low capacities while this bias is reversed at high capacities. These overall biases could be accounted for by assuming that participants have a biased model of the richness of the environment, not estimating as much as they should the extreme nature of poor or rich environments (Schustek et al., 2019).

We next studied whether the deviations from optimality weakened over time or were persistent. We fitted the power-law model to individual BD trade-off in each environment (block) for the first and second halves of each block separately (median split, Figure 4, BD trade-offs are presented in Figure 4—figure supplement 1). We observed that participants’ strategy significantly shifted towards the optimal regime from the first to the second half of the block, in the poor ($V=615,p_{adj}=.0085$) and the rich environment ($V=1651,p_{adj}=.0015$), but not in the neutral environment ($V=1271,p_{adj}=.88$). Therefore, through experience within a block, participants become closer to optimal. The magnitude of this improvement was not significantly different in the rich compared to the poor environment ($W=2222,p=.84$), suggesting that the amount of reward accumulated doesn’t influence performance.

Figure 4 with 1 supplement see all

Download asset Open asset

Finally, we wondered whether the above deviations from optimality were more pronounced in the within-subject designs (W10 and W32), where it is possible that experience on one environment carried over the next experienced environment. We observe that in several cases sampling behaviours seem to shift towards breadth in environments directly following the presentation of a poorer environment, or towards depth if a richer environment was presented immediately before (Figure 5—figure supplement 2). For instance, participants’ sampling strategy in the neutral environment (Figure 5—figure supplement 1 middle panels) seems to differ depending on whether it was presented first or not. If presented after the poor environment, we observe a clear deviation towards breadth, whereas a shift towards depth is observed if presented after the rich environment. To statistically test the presence of a sequential, or contamination effect, on participants’ sampling strategy, we compared the exponent estimated from fitting the number of observed and optimal alternatives sampled $M$ depending on the capacity using the power-law model (P) previously described (Figure 5). We observed significant positive deviations from optimality in the power factor after the presentation of a poor environment (deviation mean ± s.d.: 0.08±0.12, one-sample t-test, ${\displaystyle t_{23}=3.26,p_{adj}=.014}$), suggesting that participants sample with more breadth when previously presented with a poor environment. In contrast, we observed negative deviations after the presentation of a rich environment (–0.08±0.13, ${\displaystyle t_{23}=-2.85,p_{adj}=.036}$), suggesting that participants sample more deeply when previously presented with a rich environment. Blocks presented first (in within-subject designs or independently in between-subject designs) or after a neutral environment do not significantly deviate from optimality (first: –0.01±0.14, ${\displaystyle t_{126}=-0.81,p_{adj}=1}$, after neutral: –0.04±0.18, ${\displaystyle t_{23}=-1.13,p_{adj}=1}$). Overall, we observe a significant effect of block history on participants’ sampling strategy (measured by the difference between the observed and the optimal exponent: $a_{observed}-a_{optimal}$) (ANOVA, ${\displaystyle F_3=5.23,p=.002}$). To confirm this, post hoc analyses revealed that participants’ sampling strategies shifted towards depth in blocks presented after a poor environment compared to blocks presented after a neutral (t-test with Bonferroni correction: ${\displaystyle t_{24}=2.79,p_{adj}=.048}$) and rich environments (${\displaystyle t_{24}=4.32,p_{adj}=5.1\times {10}^{-4}}$), and compared to blocks presented first (${\displaystyle t=-3.27,p_{adj}=.014}$). No significant shift was found after being presented to a rich environment compared to blocks presented first (${\displaystyle t=2.21,p_{adj}=.20}$). Although based on exploratory analyses, these observations provide some evidence that participants’ sampling strategy was affected by the previous environment presented and that adaptation to a new environment requires to overcome the behavioural pattern implemented at earlier blocks.

Figure 5 with 2 supplements see all

Download asset Open asset

To follow up on this history effects we further investigated if this overcoming from the previous strategy happens totally, partially or at all within the timescale of a block. We divided each block in two halves as a function of trial number (median split) and submitted the data to an ANOVA with experience (first or second block half) and environment context (poor, neutral, or rich) inside both block history types (with an environment change or not). First, we found a significant interaction between block half and environment in both cases (ANOVAs, blocks no-change: $F_2=5.29,p=.0063$, change: $F_2=3.95,p=.024$) showing that participants seem to be closer to optimal in the second compared to the first half of the block (post hoc comparisons are reported in Supplementary file 2 and in Figure 5—figure supplement 2A). Additionally, when running an ANOVA over the whole data with experience, environment, and block history, we don’t observe any significant three-way interaction ($F_3=0.76,p=.52$), suggesting that the magnitude of this improvement is not statistically different depending on the block’s history. However, the contamination effects observed in blocks presented after another environment (environment change) do not seem to dissipate as a function of experience (time) within a block, because the block’s history (presented first or after a poor, neutral, or rich environment) still has a significant effect on the power factor $a$ deviation from optimality ($a_{observed}-a_{optimal})$ in the second half of the block (median split on the number of trials, ANOVA: $F_3=2.95,p=.039$, post hoc comparisons are reported in Supplementary files 3 and 4 and in Figure 5—figure supplement 2B). Therefore, the carryover effects from the previously used strategy are long-lived.

The above long timescale effects could be the result of learning or adaptation mechanisms that act on a much shorter timescale and that builds up or wanes over time. Therefore, we studied short-term effects of the outcome and choices in one trial on the choices made by the participants in the following trial. In particular, the outcome on the previous trial might affect the BD trade-offs in the subsequent trial. We classified trials within each block depending on the reward obtained in the previous trial (median split, controlled for capacity) and again tested the individual BD trade-offs by fitting the power-law model (Figure 6, BD trade-offs are presented in Figure 6—figure supplement 1). We found that participants’ sampling strategy was not significantly different following a low or high reward trial (paired Wilcoxon tests, poor: $V=1065,p_{adj}=1$, neutral: $V=1154,p_{adj}=1$, rich: $V=1324,p_{adj}=.50$).

Figure 6 with 1 supplement see all

Download asset Open asset

We then further investigated the impact of previously rewarded alternatives on the sampling in the next trial. We again split trials depending on the magnitude of the reward obtained in the previous trial (high vs. low median split, corrected for capacity). We observed a tendency for participants to resample (allocate at least one sample) the previously chosen alternative more often after high reward than low reward (probability mean ± s.d. low: 0.46±0.21, high: 0.50±0.22, paired t-test, $t_{125}=-3.5,p=6.54\times {10}^{-4}$ , Figure 7A). The size of this effect is small (Cohen’s d: $d=.31$) and is not significant when tested inside each environment individually (paired Wilcoxon tests with Bonferroni correction, poor: $V=1829.5,p=.34$, neutral: $V=1956,p=.94$, rich: $V=1911,p=.68$, Figure 7—figure supplement 1A). A similar bias was observed when considering, at trial t+1, the fraction of samples allocated in the previously selected alternative, at trial t (corrected for the capacity available) (Figure 7B). Participants resampled more (allocate more samples to) the previously chosen alternative when it had been associated with a high reward (median split, mean ± s.d. low: 0.12±0.08, high: 0.14±0.09, paired Wilcoxon test, $V=2215,p=1.39\times {10}^{-5}$). As before, the size of this effect was not large (Cohen’s d: $d=.41$) and did not survive when testing each environment individually (paired Wilcoxon tests with Bonferroni correction, poor: $V=1793,p=.24$, neutral: $V=1956,p=.94$, rich: $V=1947,p=.88$, Figure 7—figure supplement 1B).

Figure 7 with 1 supplement see all

Download asset Open asset

Despite the previous analysis showed, overall, that the magnitude of the reward obtained in the previous trial influences whether and how much the previously chosen alternative will be sampled in a subsequent trial, this bias does not affect sampling strategies (ANOVA: $F_1=.11,p=.74$) nor interacts with the effect of environment richness (between: $F_1=.74,p=.39$, within: $F_1=.50,p=.48$). It is not suboptimal either, as rewards are randomly assigned to each location at every trial. What is more, the main effect of interest, environment richness on the BD trade-off, remains significant (between: $F_1=7.43,p=.0074$, within: $F_1=33.71,p=1.72\times {10}^{-7}$) even when run only on participants displaying the bias (difference in the fraction of resampling between high and low outcome associated alternative ≥0.05, N=60, between: $F_1=4.98,p=.030$, within: $F_1=18.17,p=1.08\times {10}^{-4}$ , Figure 7C).

We also detected preferences for sampling alternatives associated with slightly less costly motor actions (see Appendix 3—figure 1) but, as those previously associated with high reward, we have no evidence showing that these biases affect optimality.

To closer characterize the behavioural patterns, we generated quantitative predictions from a variety of models, including the optimal model and several heuristics, and tested which are better able to predict the empirical observations. We chose to represent extreme behaviours, such as full-breadth ($M$ is always equal to capacity $C$) or depth ($M$ always equal to 2) as boundary models, as well as in-between behaviours which would represent the use of a trade-off between breadth and depth. Based on observations of the data and predictions (Moreno-Bote et al., 2020), we chose to model $M$ as function of $C$ using linear and power-law models. In the latter case, we used two versions of the power-law model: one where $M$ increases with the square root of capacity (exponent of 1/2) (Moreno-Bote et al., 2020), and another where the exponent was free to vary (see Materials and methods for details of the models). To model realistic behavioural patterns, we injected noise into the model predictions that corrupts the chosen number of suppliers $M$. As noise models, we used binomial and discretized Gaussian distributed noise added to the model prediction of $M$. In the second case, the noise follows a normal distribution with a standard deviation that increased linearly with capacity. This choice was made based on the empirical observation that the variance of $M$ increases linearly with capacity (Kendall’s rank correlation test, ${\displaystyle z=11.26,tau=.87,p<2.2\times {10}^{-16}}$ , see Figure 8—figure supplement 1). Data from the four experimental designs were analysed and are presented together. Fourfold cross-validated log-likelihoods (CVLL) are used to compare the models (Materials and methods; qualitatively identical results hold when using Akaike information criteria (AIC) instead, see Figure 8—figure supplements 3 and 4). Using binomial noise, the free power-law model outperforms all five other models (Figure 8A and Table 3). The optimal model, even though less efficient than the power model, is better at predicting participants’ sampling strategy than the pure breadth, depth, and square root models. The outcomes of model comparison are mostly equivalent when using the Gaussian noise model (for the results of this model, see Figure 8—figure supplement 2 and Supplementary file 5), and therefore we restrict our discussion to the binomial noise model from now on. We just note that the only divergence between the noise models appeared in the pair-wise comparisons, with the hierarchy of models’ performance being more clearly statistically established in the binomial noise model compared to the Gaussian one.

Figure 8 with 4 supplements see all

Download asset Open asset

We observed a significant interaction between the environment and the models on the CVLL (Scheirer-Ray-Hare test; $H_5=24.70,p=1.59{\times 10}^{-4}$). The depth model is better at predicting the data in the rich compared to the poor environment (Wilcoxon test with Bonferroni correction, $W_{66}=348,p_{adj}=3.99{\times 10}^{-6}$). The square root model is also shown to perform worst in the poor environment compared to the rich ($W_{66}=379,p_{adj}=1.06{\times 10}^{-5}$) and the neutral ($W_{66}=325,p_{adj}=1.88\times {10}^{-6}$) environments. Additionally, the free power-law model once again outperforms any other models in all environments (Figure 8B).

As overall the free power-law model is best at predicting the empirical data, we chose to confirm the effect of the environment on participants’ sampling strategy (see Figure 3) comparing the exponents parameters estimated from the power-law model between environments (Figure 8C). This new analysis confirmed the results reported earlier (see Figure 3), with a significant effect of the environment on the exponent $w$ in all designs B10 (Kruskal-Wallis test, ${K-W\chi }_2^2=14.43,p=7.34{\times 10}^{-4}$), W10 (one-way ANOVA, $F_1=26.47,p=8.11{\times 10}^{-5}$), and W32 ($F_1=12.81,p=.002$), except for the design B32 ($F_1=0.84,p=.37$). In all three designs W10, B10, and W32, we observed significant differences in the power exponent between the poor and the rich environment (t-tests with Bonferroni corrections, W10: $t_{17}=5.15,p_{adj}=2.43{\times 10}^{-4}$ , B10: $W_{15}=182,p_{adj}=.012$, W32: $t_{17}=3.58,p_{adj}=.007$). We also observed significant differences between the poor and the neutral environment in the designs W10 ($t_{17}=3.16,p_{adj}=.017$) and B10 ($t_{15}=200,p_{adj}=8.37\times {10}^{-4}$) but not in the design W32 ($t_{17}=2.14,p_{adj}=.142$). Moreover, we did not find any significant difference in the power exponent $w$ values between the neutral and rich environments in any of the designs (W10: $t_{17}=0.42,p_{adj}=1$, B10: $t_{15}=127,p_{adj}=1$, W32: $t_{17}=1.62,p_{adj}=.37$).

Above, we have presented analyses regarding how participants allocate the samples within the alternatives selected during sampling and compared it to optimal allocation of samples across capacities and environments. Deviations from the optimal strategy are clear, as shown both in the empirical data (which often deviates from the optimal allocation) and in the outcome of model comparison to fit these empirical data. This deviation is interesting because the difference from optimality is not exclusively accounted for by unbiased noise introduced in the model comparison above, suggesting the existence of a source of systematic bias in human choice behaviour.

In order to better understand these deviations from optimality, we looked in more detail at the most frequent observed allocation of samples. We observed that participants frequently allocated the samples homogeneously across the selected alternatives, a sampling strategy which in most cases differs largely from optimal allocation (Figure 9A). To characterize this bias towards homogenous sampling, we computed the standard deviation of the ordered counts for each allocation of samples averaged for each participant and environment (Materials and methods) and compared it to the standard deviation of the optimal allocation (Figure 9B). We observed that the standard deviation of the sample allocations measured in the poor (Wilcoxon test with Bonferroni correction, ${\displaystyle W_{66}=529,p_{adj}=7.01\times {10}^{-4}}$), neutral (${\displaystyle W_{66}=295,p_{adj}=6.87\times {10}^{-7}}$), and rich (${\displaystyle W_{66}=48,p_{adj}=4.38\times {10}^{-11}}$) environments were all significantly lower than the standard deviation assuming optimal sample allocation. Moreover, we observed that the magnitude of these differences differed between environments (Kruskal-Wallis test, ${\displaystyle K-W{\chi }_2^2=111.92,p<2.2\times {10}^{-16}}$). In fact, they increased with the environment richness, with larger differences observed in the rich compared to neutral (Wilcoxon test with Bonferroni correction, ${\displaystyle W_{66}=4089,p_{adj}=1.04\times {10}^{-17}}$) and poor (${\displaystyle W_{66}=4183,p_{adj}=2.20\times {10}^{-19}}$) environments and larger differences in the neutral compared to the poor environment (${\displaystyle W_{66}=2922,p_{adj}=.002}$). That is, there was an overall bias towards homogenous sampling, and this bias was stronger as the environment was richer.

Figure 9

Download asset Open asset

Finally, we investigated whether the observed deviations from the optimal sample allocation impacted the average reward obtained by the participants in our task. In each trial, the reward or outcome is defined as the number of good-quality apricots among the 100 apricots bought. When computing the differences between the observed and optimal outcomes (the averaged outcome obtained when following the ideal observer), we observed significantly negative deviations from optimality in all three environments (mean ± s.d. in poor: –8.74 ± 11.60%, neutral: –2.56 ± 4.23%, and rich: –2.39 ± 2.74%; two-way Wilcoxon tests with Bonferroni correction, poor: $W_{66}=311,p_{adj}=1.18\times {10}^{-6}$ , neutral: $W_{66}=457,p_{adj}=1.04\times {10}^{-4}$ , rich: $W_{66}=228,p_{adj}=6.33\times {10}^{-8}$) (Figure 9C), showing that the bias observed in the sample allocation did have a deleterious impact on the outcome. The magnitude of these differences between optimal and actual outcome was modulated by the environment (Kruskal-Wallis test, ${\displaystyle K-W{\chi }_2^2=12.29,p=.002}$), and while the harmful effect of a suboptimal strategy was moderate in the poor environment, it was very small in the remaining two. Surprisingly, the direction of the effect of environment on outcome was opposite to the one observed on standard deviations, as more deviation from optimality in the rich environment did not affect performance as much as in the poor environment. This inverted correlation is due to the nature of the environment itself. Indeed, a small deviation from the optimal strategy in the poor environment has a much larger impact on the outcome than in the rich environment, where a great majority of alternatives have a high probability of success. Indeed, we observed that the difference between the optimal and the observed outcomes seemed to decrease as the environment richness increased with differences in the poor environment being significantly higher than the ones in the neutral (Wilcoxon tests with Bonferroni correction, ${\displaystyle W_{66}=1549,p_{adj}=.013}$) and rich environment (${\displaystyle W_{66}=1475,p_{adj}=.004}$). No significant difference was observed in between the neutral and rich environments (${\displaystyle W_{66}=2185,p_{adj}=1}$).

Participants’ tendency to sample homogenously may also be one of the sources of deviation from optimality observed in the number of alternatives sampled $M$. Indeed, we observe that among the trials where participants did not use a pure breadth nor a pure depth strategy (${\displaystyle M<C}$ and ${\displaystyle M>1}$), they first allocate all the samples in an alternative before sampling another one (depth-focused) on 39.9% of the trials. In contrast, they chose to first sample once all the alternatives $M$ before adding additional samples over them (breadth-focused) on 21.8% of the trials. While the first strategy prioritizes the number of samples per alternative, the second one focuses on the number of alternatives sampled. We investigated further how these strategies prevail depending on $C$ and $M$ and observed that while the breadth-focused strategy is mostly present for trials with a shallow sample allocation ($M$ close to $C$), the depth-focused strategy is present across all types of BD trade-offs ($M/C$ ratios) (Figure 10).

Figure 10

Download asset Open asset

In the normative model the optimal sampled alternative is the alternative $i$ that maximizes the normative value ${\displaystyle V_{norm}^i=\frac{\left(\sum O_{s,i}+\alpha \right)}{N_i+\alpha +\beta }}$, where $\alpha$ and $\beta$ are the parameters describing the prior distribution of rewards in the current environment. We observed that participants select the optimal sampled alternative according to the above rule on 96.0 ± 4.34% (mean ± s.d.) of the trials. If we consider a different decision rule that selects the option with the highest proportional outcome $V_{prop}^i=\frac{\sum O_{s,i}}{N_i}$ (independently of the environment and thus independently of the parameters $\alpha$ and $\beta$), we observe that participants select the best sampled alternative on 96.6 ± 4.11% (mean ±s.d.) of the trials. This is significantly larger than when considering the normative optimal alternative (Appendix 4—figure 1), paired Wilcoxon test, ($V=454,p=6.83\times {10}^{-4}$), although very small in magnitude. Further, this effect is not homogenous amongst all environments (permutation ANOVA, $F=4.03,p=.020$) and is only significant in the poor (paired Wilcoxon test with Bonferroni correction, $V=87.5,p_{adj}=5.82\times {10}^{-4}$) and rich environments ($V=14.5,p_{adj}=4.68\times {10}^{-4}$). No significant difference is found in the neutral environment ($V=175,p_{adj}=1$). These results may also reflect difficulties for participants to estimate the relative richness of the environmental context.

In few cases (3.40 ± 4.11% of trials) when participants did not select the highest proportional alternative (the one maximizing the proportional outcome $V_{prop}$), we analysed what was the number of samples allocated in the selected alternative. We observe that, in such cases, the selected alternative had significantly more samples than the alternative maximizing the proportional outcome (mean ± s.d. of the difference in samples number: 1.78±1.51, Wilcoxon test: $V=1303,p=2.02\times {10}^{-9}$), suggesting that participants favour less uncertain alternatives.

We developed a protocol – the BD apricot task – to study how humans sample the environment using limited resources, to make economic choices. The task was programmed using Flutter development software (flutter.dev) as an application for smartphones or tablets. This allowed testing participants outside lab premises. Participants were initially introduced with a realistic narrative that provided a concrete context to aid understanding the task goals and constraints (for a demonstration video of the task, see Video 1). According to this narrative, in each trial the participant had to buy an order of apricots in bulk from one specific supplier, out of many available. The goal was to maximize the amount of good-quality apricots accumulated throughout the experiment. Because suppliers vary in the proportion of good-quality apricots they serve, participants were given the opportunity to learn about the suppliers’ overall quality by sampling: prior to the purchase, participants were asked to sample apricots from various suppliers of their choice using a fixed number of ‘free’ samples given to them. Based on this, they were to choose the supplier for the final purchase in the trial.

Specifically, each trial in the task was divided into a sampling phase and a final purchase phase. In the sampling phase (Figure 1a), the participant was given a number of coins (yellow dots) that varied from trial to trial. The number of coins determines the search capacity of the participant on each trial. We ran different designs, which varied in the range of available capacity throughout trials, with each capacity selected randomly and equiprobably, from the pre-established range. In low-capacity designs, the range of possible capacities ranged between 2 and 10, in steps of one, while in high-capacity designs capacity took one of the values in the set {2,4,8,16,32}. In each trial, the coins could be freely allocated one by one to any of the suppliers by clicking the active coin in the middle of the display and then by clicking the desired supplier to sample from. Participants could arbitrarily allocate the coins in a given trial (i.e. all coins to just one supplier, or each coin to a different supplier, or anything in between). The number of possible suppliers was always fixed to the maximum capacity of the design being tested. Once all the coins had been allocated (Figure 1b), the samples were revealed (Figure 1c) as either of good- (orange) or bad-quality (purple) apricots. The sampling outcomes $X_i$ at each supplier $i$ (given the range 1–10, or 1–32) followed a binomial distribution $X_i~B(n_i,p_i)$, where $n_i$ is the number of samples allocated in supplier $i$ and $p_i$ is the fraction of good-quality apricots in that supplier. While $n_i$ is chosen by the participants, $p_i$ is unknown to them. Based on the information collected, participants could estimate $p_i$ , and based on the estimation could choose amongst the sampled suppliers (and only the sampled ones) to perform a final bulk purchase of 100 apricots. The number of good-quality apricots contained in the purchase was revealed (Figure 1d), and the next trial (purchase cycle) started. The cumulative sum of good-quality apricots collected, as well as a bar informing participants about their progress, were displayed at the bottom of the screen throughout the experiment.

Independently in each trial and for each supplier $i$, the fraction $p_i$ of good apricots was randomly drawn from a beta distribution (with parameters α, β). We considered three different environments, varying in the relative abundance of good apricots ($p_i$), denoted poor (α=1/3, β=1), neutral (α=3, β=3), and rich (α=1, β=1/3). Participants were either presented to one environment throughout the experiment (between-subject designs) or to the three environments in different blocks of the same experiment with block order counterbalanced between participants (within-subject designs – see below for more details). In all cases, participants were verbally instructed about the relative richness of the environment they are in (poor/neutral/rich: ‘a majority of suppliers have a low/average/high proportion of good-quality apricots’) and they are aware that even though alternatives are different in each trial, they are extracted from the same environment.

Four experimental designs were run varying in the number of environments presented and the sampling capacity (see Table 1 for summary). In the two between-subject (B) designs, participants were presented only with one environment (rich, neutral, or poor), and with one of two capacities, narrow capacity (up to 10 – design B10) or wide capacity (up to 32 – design B32). In the within-subject (W) designs, participants were presented with all three environments (in blocks), either with narrow capacity (up to 10 – design W10) or with wide capacity (up to 32 – design W32). In all designs, participants were presented with eight repetitions of each sampling capacity for each environment (one or three depending on the design). All participants were first presented with a practice block composed of 10 trials covering the whole range of sampling capacities in their design (these trials were excluded from the analyses). In the middle of the practice block, participants had to take a short quiz to assess their understanding of the task. They were then provided with feedback and in case of an insufficient score (less than six correct answers out of eight questions), they had to redo the quiz for their participation to be accepted. The whole experiment was self-paced, and opportunities were given to participants to rest in halfway through and after each block.

Participants were recruited through the online platform Prolific ( 2021 Prolific, https://www.prolific.co/), with the criteria of being fluent in English, and with age between 18 and 52 years of age. They were asked to perform the task using a touch screen device (smartphone or tablet for designs with 10 alternatives, only tablet for designs with 32 alternatives). Participants received a fixed monetary compensation of 8€ per hour and, to increase their motivation, an additional reward was attributed depending on their final score at the task. In between-subject designs, participants who obtained the first and second top scores in each environment received respectively 20€ and 10€. In within-subject designs, the players with the three top scores overall were rewarded with 20€, 10€, and 5€, respectively.

Participants were recruited until completing a valid final sample size of 45 participants in each of the between-subject designs (B10 and B32 – 15 participants for each environment), and 18 participants in each of the within-subject designs (W10 and W32). The final sample size included in the study was 126 (84 males, mean age ± s.d.: 25.7±6.9).

Sample size was decided before starting data collection. In between-subject designs, sample size was calculated (using GPower, Faul et al., 2009) to achieve a 95% power in discriminating participants strategy between environments and was based on the expected effect size (estimated at Cohen’s d=0.467) of the environment on the sampling strategy (M~C). This effect size was calculated based on the expected effect of the environment following the ideal observer (see below for more details) and the variance observed within each group (s.d.=0.3) in a previous pilot study. In within-subjects design, the expected effect size was harder to estimate due to a likely contamination in participants’ strategies between environments. As we needed to counterbalance the presentation order of the three environments, the sample size had to be a multiple of 6 and thus simply we opted for 18 participants.

Data from an additional 26 participants were discarded before analysis based on pre-established criteria: the use of a wrong device (e.g. computer), insufficient score at the instructions quiz (less than six out of eight), a score not significantly higher than chance in the task, and the selection of the ‘worst supplier’ (the one presenting, after sampling, the smaller proportion of good-quality apricots) on 10% or more of the trials. We considered that such mistakes could occur when participants were confused and based their choice on the opposite colour or when they simply did not pay enough attention. As a result, three participants (in B32) were excluded because they used a wrong device, one participant (in W32) because s/he did not pass the instructions quiz, eight because of a score not significantly higher than chance (three in B10, one in W10, two in B32, and two in W32), 9 because of more than 10% ‘worst choice’ (four in B10, three in W10, and two in W32), and five because they did not satisfy the two last criteria (one in W10, four in B32).

Analyses were run using R and MATLAB. Normality of the data was tested using Shapiro tests and homoscedasticity was tested using F tests or Bartlett tests (for more than two samples). In cases where it was possible, parametric tests were preferred, otherwise non-parametric tests were used. One-sample Wilcoxon tests against the environment averaged outcome (25, 50, and 75 respectively for the poor, neutral, and rich environment) were used to test whether participant’s final score was significantly different than chance.

Our objective is to investigate how humans allocate limited resources to gather information about alternatives whose probability of success is unknown a priori. The resources are represented in our study by the number of samples available (sampling capacity, or coins) on a trial-by-trial basis. We first focus on how this limited capacity influences the number of alternatives participants chose to sample. To do so we introduce the notion of ‘sampling strategy’ characterized as the number of alternatives sampled ($M$) depending on the sampling capacity ($C$) available (Figure 2). The lower the ratios of M/C, the more the strategy tends to depth, whereas a ratio of 1 indicates pure breadth. We observe how closely empirical behaviour relates to the optimal strategy predicted theoretically (see Moreno-Bote et al., 2020) and whether it is affected by contextual parameters such as the overall probability of success (environment richness).

The optimal sampling strategy is modelled after Moreno-Bote et al., 2020, and we provide here the details. The assumptions of the framework are characterized by normative agents who don’t show any memory leak and are aware of the environment priors ($\alpha$ and $\beta$). More precisely, normative agents are set to maximize expected reward and to select the sampled alternatives which maximize the normative value ${\displaystyle V_i=\left(\sum _1^s{O}_{s,i}+\alpha \right)/\left(N_i+\alpha +\beta \right)}$ , where $O_{s,i}$ is the outcome of each sample $s$ (1 or 0) allocated in the alternative $i$, $N_i$ is the total number of samples allocated, and $\alpha$ and $\beta$ are the parameters that describe the beta distribution from where rewards in the environment are drawn. The normative strategy is described at two levels depending on both the resources available (capacity $C$) and the environment richness. Firstly, at the trial level, it predicts how many alternatives should be sampled (BD trade-off, Appendix 1—figure 1A). At low capacity (e.g. ${\displaystyle C<6}$ for the poor environment), the optimal model predicts a pure breadth, meaning that each resource sample should be allocated in a different alternative. As capacity increases, we observe an abrupt change of regime with the optimal number of sampled alternatives being close to a power law of the capacity (Moreno-Bote et al., 2020). Intuitively, when the agent has more resources, it is best to focus samples on a few alternatives, rather than spreading them over too many, as the latter strategy will allow for very little discriminability between the quality of the sampled alternatives. The capacity at which the transition between pure breadth and BD trade-off happens directly depends on the richness of the environment, the poorer the environment is and the later (at higher capacity) the transition will occur. Secondly, the normative model predicts how many samples should be allocated in each of the sampled alternatives (Appendix 1—figure 1B). In deep allocations and rich environments especially, the optimal behaviour is defined by non-homogenous allocations of samples to break ties.

We did not observe any effect of participants’ gender on the BD trade-off (ANOVA: $F_1=.21,p=.65$) nor an interaction between gender and environment richness effects (between: $F_1=.31,p=.58$, within: $F_1=.29,p=.60$). We did not observe either any effect of participants’ age (median split: 18–23 vs. 24–52 years of age) on the BD trade-off ($F_1=1.45,p=.23$). Additionally, no significant interaction between age and environment richness was found (between: $F_1=3.20,p=.076$, within: $F_1=.10,p=.78$).

In order to characterize how participants’ sampling strategies might differ depending on the richness of the environment, we decided to test three models to fit $M$ as a function of capacity $C$ separately to data from each participant and environment:

1. A piece-wise power-law model (W): ${\displaystyle M\left(C\right)=\left\{\begin{array}{c}{C}^{a_1}ifC\le B\\ C^{a_2}+bifC>B\end{array}\right\}}$, where $B$ corresponds to the breakpoint with $B\in \{3,4,\dots ,9\}$ in narrow-capacity designs, and $B\in \{3,4,\dots ,16\}$ in wide-capacity designs.

2. A linear model (L): $M\left(C\right)=aC+b$.

3. A power-law model (P): $M\left(C\right)=C^a$.

Linear and power-law models were compared using a paired Wilcoxon test on individual R-squared adjusted while power-law and piece-wise power law models were compared using ANOVA (with ${\displaystyle \alpha =.05}$) at the participant and environment levels. The power-law model captured the empirical relationship between $C$ and $M$ best.

Given the results of the model comparisons, the effect of the environment on participants’ sampling strategies was therefore tested using the power-law model. We compared the power factor $a$ extracted from power-law fits in each environment using within (designs W10 and W32) or between (designs B10 and B32) one-way ANOVA. Post hoc comparisons between environments were conducted using t-tests with Bonferroni correction for multiple comparisons. The four experimental designs were analysed individually, in order to account for the structure of the data (within- or between-subjects).

Fitting $M$ as a function of capacity, we previously observed that even when participants’ sampling strategy falls close to the predicted optimal one (see Figure 2), they do not coincide completely. To better characterize participants’ deviations from optimality we first describe them overall, independently from the capacity, by fitting the optimal number of alternatives sampled ($M_{opt}$) as a function of capacity using the power-law model (described above) as it was the one better accounting for our data. We compared both power factors $a$ using one-sample t-tests in each environment individually, correcting for multiple comparisons, and observed significant differences, going in counter-directions depending on the environment. To further understand these deviations, we investigate if capacity could be a cause of modulations. To do so, we computed the differences between the number of alternatives observed $M$ and $M_{opt}$ for each capacity and environment (Appendix 2—figure 1) and tested for marginal effects of the capacity and the environment, as well as for an interaction between capacity and environment using a Sheirer-Ray-Hare test. All effects were found significant and post hoc analyses were performed using one-sample Wilcoxon tests (against the null hypothesis $mu=0$, see Supplementary file 1) in order to better understand how both capacity and environment richness affect the nature and directions of participants’ sampling deviations from optimality.

The BD dilemma can be solved using extreme behaviours (pure-depth or pure-breadth) but also trade-offs which establish a balance between breadth and depth and often lead to more optimal choices. In the current study, we quantitatively assess which strategy describes best the individual participants’ sampling behaviour by comparing the ability of the optimal and heuristic models (characterizing both extreme and trade-off behaviours) to predict the data. The number of vendors $M$ that participant k chose in trial j, denoted $M_j^k$ , was predicted by using one of three models, corrupted by either Gaussian or binomial noise. Each model predicts the mean $M\left(C_j^k\right)$ number of selected vendors given that capacity was $C_j^k$ . We use the following models:

1. Optimal model: $M\left(C\right)$ follows the ideal observer $M_{opt}\left(C\right)$

   •      ${\displaystyle M\left(C\right)=M_{opt}\left(C\right)}$.

   • The ideal observer corresponds to the sample allocation that would maximize expected reward for each capacity in each environment. It describes the optimal number of suppliers to be sampled M, as well as the distribution of samples per supplier. The ideal observer was described in a previous theoretical paper (Moreno-Bote et al., 2020) and here has been calculated with Monte-Carlo simulations.

2. Linear models: We used three variants of linear model:

   1. Depth model: $M\left(C\right)$ is constant and independent of capacity $C$, with $M$ following an almost pure-depth strategy,

      •      ${\displaystyle M\left(C\right)=2.}$

   2. Pure-breadth model: always equals to ,

      •      ${\displaystyle M\left(C\right)=C.}$

   3. Free-slope linear model: $M\left(C\right)$ increases with capacity $C$ with a free factor $d$,

      •      ${\displaystyle M\left(C\right)=dC.}$

3. Power-law model: We used two variants of this model:

   1. Square root model: $M\left(C\right)$ increases with the square root of capacity $C$.

      •      ${\displaystyle M\left(C\right)=\sqrt{C}.}$

   2. Free power-law model: $M\left(C\right)$ increases with capacity $C$ to the $w$ th power,

      •      ${\displaystyle M\left(C\right)=C^w.}$

For each model, to compute the likelihood of the data $M_j^k$ for each trial j and participant k, we assume a noise model that is either Gaussian or binomial. In the Gaussian noise model, we assume that $M_j^k={M(C}_j^k)+{\sigma }_j^k{z}_j^k$ , where $z_j^k$ is independently and identically distributed standard normal noise, and ${\sigma }_j^k=a^k+b^k{C}_j^k$ . In all cases we obtained that the best fit parameters obeyed $a^k\ne 0,b^k\ne 0$, so that ${\displaystyle {\sigma }_j^k\ne 0and{\left({\sigma }_j^k\right)}^2>0}$ for all values of $C$. The per-participant likelihood of the model is the sum of individual trial-by-trial likelihoods

In the binomial case, we assume that the observed $M_j^k$ follows a binomial distribution $M_j^k\sim B\left(n^k,p^k\right)$ with $p^k\in \left[\mathrm{0,1}\right]$ and

so that the expectation of $M$ equals the desired model’s expectation $M\left(C\right)$. In this case, the per-participant likelihood of the data under the model and binomial noise is

The overall log likelihood of each model was the sum of log likelihoods across trials, participants, and environments.

Each model (1–3 above) with each noise model (Gaussian or binomial) was fit and tested to each participant and environment (in the case of within-subjects designs) individually using fourfold cross-validation. Specifically, in onefold, 75% of the data in each participant was used to fit the per-participant model parameters by maximizing their individual log likelihoods. The remaining 25% of the data was used to measure the quality of the fit. This was quantified as the log likelihood of the model with its fitted parameters in the test set, and summed across participants, and averaged over the fourfolds (four different ways of taking 25% non-overlapping trials for testing and the remaining for training). We call the resulting average CVLL. In the binomial noise model, the log likelihood of the test set becomes minus infinity in trials when ${\displaystyle n<M}$, which is an impossible event. This concerned a small set of trials in 27 participants in the square root model, 31 in the optimal model, 16 in the depth model, 69 in the free power-law model, and 48 in the linear model. However, in all cases it affected only one out of the fourfolds, so the averaged log likelihood could be computed across the three remaining folds. Model comparison was performed by comparing the CVLL across models and noise models using Wilcoxon matched pairs signed-ranks tests with Bonferroni correction. In order to ascertain the results of the log likelihood-based model comparisons, the models were also compared using AIC (Akaike, 1977), which were computed on the whole dataset. Both methods provide qualitatively identical results (compare Figure 8 with Figure 8—figure supplement 3 for binomial noise models and Figure 8—figure supplement 1 with Figure 8—figure supplement 4 for Gaussian noise models). Free parameter $a$ was optimized on a grid from –3 to –1 and $b$ on a grid from –1 to –1, both with a step of 0.05 in the Gaussian noise model and the parameter $p$ in the binomial noise model which was optimized on grid from 0 to 1 with a step of 0.01. The free parameter $w$ was optimized on a grid from 0.05 to 1.1 with a step of 0.05 in both Gaussian and binomial noise models.

To descript more precisely participants’ sampling strategy, we studied how the samples are allocated inside the selected suppliers. Indeed, for an identical number of alternatives sampled $M$ and capacity $C$, we can observe various allocations of samples. For example, $M=2$ and $C=4$ can result in the allocation of 2 samples in two different alternatives each; {2,2}, or in the allocation of 3 samples in a first alternative and 1 sample in a second; {3,1}. Visually, we observe that participants have a tendency to allocate the samples homogeneously across the sampled alternatives (e.g. {2,2}) (see Figure 9A). In order to statistically capture this putative bias, we computed the standard deviations of the sample allocations. An homogenous allocation of samples would result in a standard deviations close to 0 (e.g. s.d.({2,2})=0) whereas a more heterogeneous allocation of samples would be associated with higher standard deviations (e.g. s.d.({1,3}) ≈ 1.41). We computed the standard deviation of each sample allocation and compared it to the standard deviation of the optimal allocation of samples (predicted by Moreno-Bote et al., 2020) in each environment using Wilcoxon tests with Bonferroni corrections to adjust for the multiple comparisons. We tested the effect of the environment on the magnitude of these differences using a Kruskal-Wallis test and performed post hoc analyses using Wilcoxon tests with Bonferroni corrections.

We also computed the differences between the outcomes observed in our data with the ones obtained when following the ideal observer strategy. A Kruskal test was performed to test the significant effect of the environment on the outcome differences and one-sample Wilcoxon tests with Bonferroni corrections were then used to assess whether observed outcomes were significantly different than optimal outcomes inside each environment condition.

These analyses were performed for all trials (from all experimental designs) where the optimal number of alternatives sampled was strictly inferior to the capacity (${\displaystyle M_{opt}<C}$) (see Figure 9B–C). This condition has been introduced because when $M_{opt}=C$, the optimal sample allocation follows a pure-breadth strategy, and its associated standard deviation is 0. Therefore, the standard deviation of the sample allocation observed in the data can only be superior to the optimal sample allocation which impairs the observation of participants’ bias to sample homogenously the alternatives.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

This work is supported by the Howard Hughes Medical Institute (HHMI, Ref: 55008742), MINECO (Spain; BFU2017-85936-P), ICREA Academia (2016) and Ministerio de Ciencia e Innovación (Ref: PID2020-114196GB-I00/AEI) to RM-B. SS-. is funded by Ministerio de Ciencia e Innovación (Ref: PID2019-108531GB-I00 AEI/FEDER) and the FEDER/ERDF Operative Programme for Catalunya 2014-2020. AV is supported by a FI fellowship from the AGAUR (2019FI_B 00302). We would like to thank Pierre Marais for his precious help in setting up the Flutter application and coding the experimental task.

Human subjects: Before starting the experiment, participants had to give their informed consent. This study was part of the project 'IMC: INTEGRACIÓN MULTISENSORIAL Y CONFLICTO' (PID2019-108531GB-I00) for which an ethical approval was obtained.

© 2022, Vidal et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.