Adaptive behaviour in complex environments requires an intricate balance of deliberate action and efficient responding. For instance, in the supermarket faced with numerous products, it may certainly be helpful to weigh the pros and cons of a specific product before making a choice. Yet, thinking for hours about which toothpaste to buy may interfere with the goal of being home before dinner. The balance of thorough deliberation and efficiency is supported by at least two systems of behavioural control that operate in parallel: (i) a goal-directed system that learns when to perform the action required to achieve a desired outcome (in the form of stimulus [S] – response [R] – outcome [O] associations) and (ii) a habitual system that acquires S-R associations without any links to the response related outcome (Adams, 1982; Adams and Dickinson, 1981; Balleine and Dickinson, 1998). These two systems are known to rely on distinct neural circuits. While the goal-directed system primarily relies on the medial prefrontal and orbitofrontal cortex as well as the dorsomedial striatum (anterior caudate in humans; Balleine and O’Doherty, 2010), the habitual system depends on the dorsolateral striatum (posterior lateral putamen in humans; Balleine and Dickinson, 1998; Balleine and O’Doherty, 2010; Corbit and Balleine, 2003; Ostlund and Balleine, 2005; Tricomi et al., 2009; Valentin et al., 2007; Yin et al., 2006). Moreover, it is commonly assumed that the goal-directed system guides early learning, whereas the habitual system takes over once a behaviour has been frequently repeated. Therefore, while buying toothpaste for the first time should be a goal-directed action, this choice should become more habitual if we have bought the specific toothpaste several times before. Adaptive behaviour requires the capacity to flexibly switch back from habitual to goal-directed control in response to environmental changes (e.g. when the previously bought toothpaste is out of stock). Lacking this flexibility in behavioural control may be detrimental to mental health. In particular, overreliance on habitual responding has been linked to several mental disorders, including drug addiction, obsessive-compulsive disorder, schizophrenia, eating disorders, and depression (Everitt and Robbins, 2005; Voon et al., 2015; Griffiths et al., 2014; Robbins et al., 2012; Voon et al., 2015).

Accumulating evidence indicates that stressful events may tip the balance from goal-directed to habitual control. Specifically, stress (and major stress mediators) has been shown to induce a shift from goal-directed action to habitual responding (Braun and Hauber, 2013; Dias-Ferreira et al., 2009; Gourley et al., 2012; Schwabe et al., 2012; Schwabe and Wolf, 2009; Schwabe and Wolf, 2010; Schwabe and Wolf, 2013; Smeets et al., 2019; Soares et al., 2012). Beyond its crucial relevance for our understanding of behavioural control in general, this stress-induced shift towards the habitual system may be a driving force in mental disorders that are characterized by dysfunctional stress responses on the one hand and aberrant habitual control on the other (Adams et al., 2018; Goeders, 2004; Schwabe et al., 2011a). Although the stress-induced bias to habitual responding may have critical clinical implications, the exact mechanisms through which stress modulates the balance of goal-directed and habitual control are not fully understood. In particular, previous research has been unable to address the fundamental question of whether the stress-induced bias towards habitual behaviour is due to a downregulation of the goal-directed system or the enhancement of the habit system, or both.

Canonical assays for the assessment of goal-directed and habitual control do not allow a distinction between these alternatives. These paradigms are based on the key distinctive feature of goal-directed and habitual control, that is, only the goal-directed system is sensitive to changes in the motivational value of the outcome in absence of new experience with it (Adams, 1982; Dickinson and Balleine, 1994). Accordingly, classical paradigms have tested the behavioural sensitivity to either a devaluation of an outcome or to the degradation of the action-outcome contingency (Adams, 1982; Corbit and Balleine, 2003; Tanaka et al., 2008; Valentin et al., 2007; Yin et al., 2004). Although these elegant paradigms provide valuable insight into the mechanisms involved in behavioural control, they are unable to determine whether increased responding to devalued or degraded actions is due to reduced outcome or enhanced S-R processing, or both.

Here, we aimed to overcome these shortcomings of classical paradigms that are used to determine these modes of behavioural control and to examine whether stress leads to an upregulation of response processing at the time of stimulus presentation or a downregulation of outcome-related processing, or both. To these ends, we leveraged EEG in combination with multivariate pattern analysis (MVPA)-based decoding of neural representations. In the present study, we first exposed participants to a stress or control manipulation and then asked them to complete an instrumental learning task during which they could learn S-R(-O) associations. Crucially, we used image categories as response options (R) and outcome categories (O) that have a distinct neural signature and recorded EEG throughout the task. EEG-based classifiers (support vector machines, SVMs) were trained to distinguish between the R and O stimulus categories on a separate delayed-matching-to-sample (DMS) task. We then applied these classifiers to the instrumental learning task to decode the outcome representations relevant to the goal-directed system when participants saw the stimulus S and when they made the response R. Furthermore, we decoded response R representations when participants saw the stimulus S, pointing to existing S-R representations (which may, however, be relevant for both habitual S-R learning and goal-directed S-R-O learning). Using a similar decoding approach on functional magnetic resonance imaging (fMRI) MRI data, a previous study showed that brain regions implicated in goal-directed control contained information about outcomes and responses, whereas regions associated with habitual responding contained only information about responses (but not outcomes) at the time of stimulus presentation (McNamee et al., 2015).

Because previous rodent studies suggested that stress or stress hormone effects on the balance of goal-directed and habitual forms of learning are training-dependent (Dias-Ferreira et al., 2009; Packard, 1999; Siller-Pérez et al., 2017), we also assessed training-dependent dynamics in the stress effect on outcome and response processing. Although we did not aim to test overtraining-induced changes in the balance of outcome and response processing, for which the number of trials may have been too limited in the present study, we included transient outcome devaluations at the beginning, middle, and end of the instrumental learning task to assess whether stress effects on instrumental behaviour are training-dependent. These outcome devaluations were included in order to assess if and how the predicted changes in neural representations are linked to behavioural manifestations of stress-induced changes in behavioural control.

The goal of this study was to elucidate the mechanisms underlying the impact of stress on the control of instrumental behaviour. Specifically, we aimed to leverage an EEG-based decoding approach to determine stress-induced changes in outcome and response representations that constitute key features that distinguish goal-directed action and habitual responding (Adams, 1982; Adams and Dickinson, 1981; Balleine and Dickinson, 1998; Balleine and O’Doherty, 2010). To this end, participants first underwent the Trier Social Stress Test (TSST; Kirschbaum et al., 1993), a mock job interview that represents a gold standard in experimental stress research (Allen et al., 2014), or a non-stressful control manipulation. Afterwards, participants completed a reinforcement learning task (Luque et al., 2017) that allowed us to probe the goal-directed vs. habitual control of behaviour. In this task, participants could learn S-R-O associations (Figure 1). Specifically, they acted as ‘space traders’ who traded two cards, represented by two distinct fractals (green vs. pink), with two alien tribes that were represented by distinct symbols (red vs. blue) in return for cards from two possible categories (objects vs. scenes). On each trial, participants first saw one of the two fractals (S). They were then shown representatives of the two alien tribes next to each other and asked to decide which alien to offer the fractal to (R). Finally, participants received feedback about whether the alien accepted the offer and traded one of the desired cards and how many respective points were earned (O). Importantly, one alien tribe accepted only one type of fractal and traded only one card category. Furthermore, one card category was worth more than the other (high-valued outcome, O^high, and low-valued outcome, O^low). Participants had to learn these associations using trial-by-trial feedback. Moreover, there was a response cost associated with each trade that was accepted by the alien.

Figure 1

Download asset Open asset

This task can be solved by ‘goal-directed’ action-outcome (S-R-O) learning and by ‘habitual’ S-R learning. In order to reveal the mode of control at the behavioural level, we presented task blocks in which one of the outcomes was devalued (i.e. not worth any points but associated with a response cost, thus resulting in a negative outcome). If behaviour is goal-directed, it should be sensitive to the outcome-devaluation, and participants should avoid the devalued action. Conversely, if behaviour is habitual, it should be less sensitive to the outcome devaluation, and participants should be more prone to perform the frequently repeated but now devalued response. To assess whether the balance of goal-directed and habitual behaviour and its modulation by stress is training-dependent, we presented devaluation blocks early during training, after moderate training, and again after extended training at the end of the task.

Critically, we recorded EEG during task performance and used stimulus categories as R and O stimuli that are known to have distinct neural signatures (Bae and Luck, 2018; Cairney et al., 2018; Taghizadeh-Sarabi et al., 2015; Treder, 2020). We trained EEG-based multivariate classifiers in an unrelated DMS task to discriminate between stimulus categories that were used as S (blue vs. red symbols that differed also in shape and line orientation) and O (objects vs. scenes) (Figure 1). The DMS task was completed both before and after the reinforcement learning task, and the classifier was trained on the pooled trials of the two DMS sessions (thus ruling out any time-dependent biases of the classifiers). The trained classifiers were then applied to the reinforcement learning task to determine neural representations of R and O.

Response representations were decoded during the S presentation, whereas O representations were decoded during both the S presentation and participants’ choice (R). Outcome representations at the time of S presentation and R are indicative of goal-directed control. In contrast, response representations at the time of stimulus-representations may be relevant for both goal-directed S-R-O learning and habitual S-R learning.

In order to adequately assess whether increased responding to devalued actions is due to reduced goal-directed or enhanced habitual control, or both, we employed a variety of measures and statistical approaches. At the behavioural level, we applied a devaluation paradigm to examine whether stressed participants respond differently to transient outcome devaluations compared to control participants and to assess whether the predicted neural changes are linked to behavioural outcomes. Behavioural analyses focussed mainly on participants’ choice in valued and devalued trials. Based on previous animal studies (Dias-Ferreira et al., 2009), we additionally investigated whether the effects of stress on the mode of control would be training-dependent and therefore implemented outcome devaluation at the beginning, middle, and end of the instrumental task. At the physiological level, we analysed pulse, systolic and diastolic blood pressure as well as salivary cortisol concentrations at different time points during the experiment to assess for the effectiveness of the stress manipulation. At the neural level, we mainly focused on the decoding of outcome and response representations, which provide insights into the mechanisms through which stress affects sensitivity to outcome devaluation - the primary objective of the present study.

To further address the important question of whether response and outcome representations reflect signatures of distinct control systems, we additionally analysed the correlations between both neural representations. For this purpose, we used Bayesian correlational analyses. Depending on the magnitude of the Bayes factor (reflecting the likelihood ratio of the data under the alternative hypothesis and the data under the null hypothesis), the Bayesian approach can provide evidence in favour of the alternative hypothesis or evidence in favour of the null (Hoijtink, 2012; Kass and Raftery, 1995; Nuzzo, 2017). Thus, we utilized Bayesian analyses to provide clear evidence for or against the null hypothesis. To further assess the association between the behavioural data (i.e. classification accuracy) and the strength of the neural representation (i.e. classification accuracies), we computed Spearman correlations. In order to also analyse previously proposed ‘attentional habits’ (Luque et al., 2017), we analysed event-related potentials (ERPs).

Successful stress manipulation

Significant subjective and physiological changes in response to the TSST confirmed that stress was successfully induced. Compared to participants in the control group, participants exposed to the TSST experienced the treatment as significantly more stressful, difficult, and unpleasant than those in the control condition (all t₅₆ >5.82, all p<0.001, all d>1.530, and all 95% CI=1.456–2.112; Table 1). At the physiological level, exposure to the TSST elicited significant increases in pulse and systolic and diastolic blood pressure (time point of measurement × group interaction, all F[4, 224]>13.55, all p<0.001, all ƞ_p²>0.195, and all 95% CI=0.100–0.425; Figure 2). As shown in Figure 2A–C, although both groups had comparable blood pressure and pulse before and after the TSST (all t₅₆ <2.28, all p_corr>0.081, all d<0.600, and all 95% CI=0.069–0.702), participants in the stress group had significantly higher blood pressure and pulse during the experimental manipulation than those in the control group (all t₅₆ >4.21, all p_corr<0.001, all d>1.107, and all 95% CI=0.549–2.098). Finally, salivary cortisol concentrations increased in response to the TSST but not after the control manipulation (time point of measurement × group interaction: F[3, 168]=6.69, p<0.001, ƞ_p²=0.107, and 95% CI=0.026–0.188). As shown in Figure 2D, participants in the stress and control groups had comparable cortisol concentrations at baseline (t₅₆=1.16, p_corr=1, d=0.304, and 95% CI=−0.216–0.821). However, about 20 min after the treatment, when the reinforcement learning task started, cortisol levels were significantly higher in the stress group than in the control group (t₅₆=2.74, p_corr = 0.032, d=0.720, and 95% CI=0.185–1.249). As expected, cortisol levels returned to the level of the control group by the end of the task (60 min: t₅₆=0.50, p_corr = 1, d=0.130, and 95% CI=−0.386–0.645; 105 min: t₅₆=0.42, p_corr=1, d=0.111, and 95% CI=−0.405–0.625).

Table 1

	Control		Stress
	M	SEM	M	SEM
Subjective assessments
Stressfulness	16.79	3.45	62.33*	4.44
Unpleasantness	15.36	3.51	57.00*	5.90
Difficulty	14.29	3.35	51.33*	5.29

1. Subjective assessments were rated on a scale from 0 (‘not at all’) to 100 (‘very much’). *p<0.001, Bonferroni-corrected, significant group difference.

Figure 2 with 4 supplements see all

Download asset Open asset

Figure 3 with 2 supplements see all

Download asset Open asset

Stress reduces outcome representations at the end of training

Thus far, our behavioural data showed that stress rendered behaviour less sensitive to a change in outcome value, which can be interpreted as decreased goal-directed or increased habitual behaviour. In addition, stress reduced electrophysiological late latency potentials that appeared to be sensitive to the value of an outcome (Luque et al., 2017).

In a next step, we leveraged an EEG-based decoding approach to address the primary objective of this study, that is, to probe the effect of stress on neural outcome and response representations. We trained an MVPA classifier (SVM) based on an independent dataset (DMS task, for details see Materials and methods) to discriminate between categories that were used during the reinforcement learning task as an outcome (object card vs. scene card). This classifier was then used to assess trial-by-trial changes in outcome representations throughout the reinforcement learning task. Goal-directed control should be reflected in high accuracy of outcome category classification during the presentation of the stimulus (S) as well as during the response choice (R).

We first analysed outcome representations during the presentation of the fractal stimulus S before participants had to make a choice. This analysis revealed a significant block × group interaction (F[3, 117] = 2.77, p=0.045, ƞ_p²=0.066, and 95% CI=0–0.149). As training proceeded, participants in the stress group showed a reduced outcome representation at the time of S presentation (first six vs. last six blocks: t₂₂=3.59, p_corr=0.004, d=0.748, and 95% CI=0.277–1.206), whereas the outcome representation remained rather constant in participants of the control group (first six vs. last six blocks: t₁₇=1.08, p_corr = 0.590, d=0.255, and 95% CI=−0.219–0.721; Figure 2F). In the last six blocks of the task, the classification accuracy for the outcome was significantly lower in the stress group compared to the control group (t₃₉=3.13, p_corr=0.012, d=0.986, and 95% CI=0.326–1.635; lower training intensity [first 18 blocks of the task]: all t₃₉<1.13, all p_corr=1, all d<0.355, and all 95% CI=−0.914–0.975; note that the overall pattern remains when trials are blocked differently, see Supplementary file 1A). Strikingly, the reduced outcome representation was significantly correlated with the reduced behavioural sensitivity to the outcome devaluation during Dev O^high blocks (Spearman’s ρ=–0.482, 95% CI=−0.688−0.205, and p=0.001; Figure 4).

Figure 4

Download asset Open asset

Analysing the outcome representations at the time of choice between the two aliens revealed a very similar pattern; at the end of the task, participants in the stress group showed a decreased outcome representation at the time point of the choice (t₂₂=2.94, p_corr=0.016, d=0.613, and 95% CI=0.161–1.054), whereas there was no such effect in the control group (t₁₇=0.59, p_corr=1, d=0.138, and 95% CI=−0.328–0.600). In addition, stressed participants had reduced outcome representations relative to controls, reflected in a significantly reduced classification accuracy, during the response choice at the end of the reinforcement learning task (block × group interaction: F[3, 117] = 2.99, p=0.034, ƞ_p²=0.071, and 95% CI=0–0.156; stress vs. control, high training intensity: t₃₉=2.75, p_corr=0.036, d=0.865, and 95% CI=0.214–1.506; lower training intensities: all t₃₉<2.30, all p_corr>0.108, all d<0.725, and all 95% CI=−0.500–1.358; Figure 2G). Together, these results show that at the end of training, acute stress reduced the representation of action outcomes that are considered to be a hallmark of goal-directed control.

Outcome and response representations are uncorrelated

To test whether the observed opposing changes in outcome and response representations after stress reflected independent or linked neural representations, we analysed Bayesian correlation between the classification accuracies in order to explicitly test the evidence in favour of the null and alternative hypothesis, respectively. These analyses revealed moderate evidence for the null hypothesis that outcome representations, both at stimulus presentation and response selection, were uncorrelated with response representation at choice time (both Pearson’s |r|<0.165, both 95% CI=0.154–0.442, and both BF₀₁ >3.092; Figure 5), suggesting that outcome representations and response representations may be independent of each other.

Figure 5

Download asset Open asset

Previous research showed that stress favours habitual responding over goal-directed action (Braun and Hauber, 2013; Dias-Ferreira et al., 2009; Gourley et al., 2012; Schwabe et al., 2009; Schwabe et al., 2010; Schwabe et al., 2011b; Schwabe et al., 2012; Seehagen et al., 2015; Smeets et al., 2019; Smeets and Quaedflieg, 2019; Soares et al., 2012; Wirz et al., 2018). Although this stress-induced bias towards habitual behaviour has important implications for stress-related mental disorders (Adams et al., 2018; Goeders, 2004; Schwabe et al., 2011a), a fundamental question has remained elusive thus far: is the shift towards habitual behaviour under stress due to diminished goal-directed control or enhanced habitual control, or both? Canonical behavioural assays of the mode of behavioural control cannot distinguish between these alternatives. Here, we used EEG-based decoding of outcome and response representations - the key components of goal-directed and habitual processing - to provide evidence that acute stress results in a decrease of outcome-related processing that is critical for goal-directed control, and paralleled by an increase in response processing.

Our behavioural and ERP data corroborates previous reports of a stress-induced shift from goal-directed to habitual control (Braun and Hauber, 2013; Schwabe et al., 2011b; Smeets and Quaedflieg, 2019). Specifically, stressed participants showed increased responding to a devalued action after extended training, suggesting a reduced behavioural sensitivity to the outcome devaluation which indicates less goal-directed behaviour and more habitual responding (Adams and Dickinson, 1981). Recent evidence showed that goal-directed and habitual processing may also be reflected in ERPs that are either sensitive or insensitive to changes in the value of an action outcome (Luque et al., 2017). We observed here that stress appears to reduce the sensitivity of late potentials to the outcome devaluation, whereas stress left the occipital P1 component that was insensitive to the outcome devaluation unaffected and may thus be considered habitual (Luque et al., 2017). However, similar to the behavioural insensitivity to an outcome devaluation after stress, these ERPs cannot separate reduced goal-directed from increased habitual responding. To disentangle outcome-related processing that is critical for goal-directed learning and S-R processing, which may be relevant both for goal-directed and habitual learning, we used an MVPA-based decoding approach. Critically, our Bayesian analyses indicated that changes in outcome and response representations were uncorrelated, which may be taken as evidence that they do not reflect changes in a single system, but rather (at least partly) dissociable signatures of goal-directed and habitual processing.

We show that stress led to a reduction in outcome representations both at the time of stimulus presentation and at the time of action selection, as well as a parallel increase of response representations during stimulus presentation indicative of enhanced S-R associations (Figure 2F–H). Both the stress-induced reduction in outcome representations and the increase in response representations were directly correlated with the behavioural (in)sensitivity to the outcome devaluation (Figure 4). However, while outcome representations were negatively correlated with participants’ responding to the devalued action, there was a positive correlation for response representations which might lend further support to the view that these representations reflect distinct processes. Taken together, our results indicate that acute stress leads to enhanced S-R processing and impaired outcome-related processing. The latter is in line with evidence showing reduced orbitofrontal activity in the face of elevated stress hormones (Schwabe et al., 2012), which is accompanied by increased habitual processing. Based on previous pharmacological studies (Gourley et al., 2012; Schwabe et al., 2010; Schwabe et al., 2011b; Schwabe et al., 2012), we assume that these opposing effects of stress on outcome-related processing and S-R processing are based on the concerted action of glucocorticoids and noradrenaline on the neural substrates of goal-directed and habitual control.

Importantly, at the time when stress effects on devaluation sensitivity and outcome representation were observed, stress mediators were no longer elevated. However, it is important to note that stress effects do not necessarily terminate when acute levels of stress mediators returned to baseline. Specifically, several major stress mediators are known to have ‘after-effects’ that outlast acute elevations of these measures (Joëls and Baram, 2009). For example, glucocorticoids are assumed to act as a ‘two-stage rocket’, with rapid, non-genomic actions and delayed, genomic actions (Joëls et al., 2012). The latter genomic actions typically set in after acute glucocorticoid elevations vanished (de Kloet et al., 2008). Thus, the fact that autonomic measures and cortisol levels returned to baseline during the learning task does not imply that the stress system activation had been over at this point. Moreover, acutely elevated stress mediators may have affected early learning processes in a way that became apparent only as training progressed.

Importantly, both our behavioural and our neural decoding data showed that stress affected the balance of outcome-related and S-R processes in a training-dependent manner. Previous human studies could not distinguish between such early and late effects of stress because the mode of behavioural control was only assessed at the end of training (Schwabe and Wolf, 2009; Smeets et al., 2019). The present results compliment rodent data which shows that (chronic) stress effects on the control of instrumental learning are training-dependent (Dias-Ferreira et al., 2009). In general, it is commonly assumed that early training falls under goal-directed control, while extensive training results in habitual control (Adams, 1982; Dickinson et al., 1995; Tricomi et al., 2009). However, whether or not overtraining may induce habitual behaviour in humans is currently debated (de Wit et al., 2018), and our data cannot speak to this issue as training may have been too limited to result in overtraining-related habits (which might require thousands of responses; Tricomi et al., 2009). Thus, training-dependent effects do not necessarily imply overtraining effects. However, findings of ‘cognitive’ and ‘habitual’ forms of navigational learning in rats demonstrated that stress hormones may accelerate a shift from ‘cognitive’ to ‘habitual’ learning that would otherwise only occur after extended training (Packard, 1999; Siller-Pérez et al., 2017). Thus, it is tempting to hypothesize that a similar mechanism might be at work during instrumental learning in stressed humans. This conclusion, however, remains speculative as we did not observe a training-dependent shift towards habitual control in the control group, and this group even showed reduced response and increased outcome representations over time, which rather suggests increased goal-directed processing across the task. Future studies are required to test this hypothesis by using the present neural decoding approach in combination with an extensive training protocol and groups of participants that are exposed to stress at distinct stages of the learning process.

In animal studies, devaluation sensitivity is usually assessed by means of a reinforcement test, in which the devalued outcome is delivered contingent on the relevant response. This procedure allows to control, for example, for a general lack of attention. In the present human study, the feedback provided on each trial varied between blocks. During NoDev blocks, participants saw the value of the card that was gained before as well as the corresponding response costs. In Dev blocks, however, this information was masked for all trials. This procedure is comparable to previous studies in humans in which the devalued outcome was not presented during critical test trials (e.g. Schwabe and Wolf, 2009; Valentin et al., 2007). Importantly, however, we also included consumption trials in all of the blocks which enabled us to rule out unspecific factors, such as a general lack of attention, altered contingency knowledge, or response perseveration. In these consumption trials, participants clearly preferred the high-valued cards over low-valued cards during NoDev blocks as well as the valued card over its devalued counterpart during Dev blocks, irrespective of stress. This finding demonstrates that participants in both groups were aware of the value of the card stimuli in a specific block but that responses of stressed participants were less guided by this knowledge about the value of the outcome engendered by the response. In addition, the specific response pattern in the consumption trials during devalued blocks also rules out general attentional or motivational deficits.

It is important to note that participants received an error feedback in devalued trials when they chose the response option that was not associated with the now devalued outcome. Given that acute stress may increase sensitivity to social cues (Domes and Zimmer, 2019), one might argue that stressed participants continued to respond towards devalued outcomes in order to avoid being presented with the error-feedback screen. However, we consider this alternative to be unlikely. First, with respect to the neural outcome and response representations, these were analysed during NoDev blocks in which groups did not differ in their behavioural performance accuracy and consequently not in the frequency of error feedback. Furthermore, participants’ performance in devalued blocks was directly associated with the observed changes in neural outcomes and response representations during the NoDev blocks, which again, could not be biased by differences in error feedback processing.

In addition, stressed participants showed an increase in insensitivity to outcome in Dev O^high but not in Dev O^low trials. Moreover, we found that the devaluation effect for O^high stimuli was stronger compared to the effect for O^low stimuli. This difference remained even when we analysed only the control group, excluding the possibility that the difference between O^low and O^high was merely due to the fact that stress increased specifically the insensitivity to the devaluation of O^high. However, why may the devaluation effect be lower for O^high than for O^low and why may stress have affected primarily the devaluation of O^high? These results suggest a stronger habit formation for stimuli that were paired with high rewards. A potential explanation for this pattern takes the links between rewards, stimulus saliency, and strength of S-R associations into account; the initial association with high valued outcomes may have increased the salience of the respective stimuli, which in turn may have promoted the formation of S-R associations. These stronger S-R associations may have resulted in more habitual responses for the devalued outcomes.

Interestingly, participants either primarily selected their action based on the outcome value (i.e. by avoiding the devalued outcome and thus were goal-directed) or responded to the devalued outcome in the vast majority of the trials and thus behaved habitually. Thus, participants showed an ‘either-or’ pattern, and there seemed to be interindividual differences in the tendency to perform in a goal-directed vs. habitual manner. Interestingly, also among stressed participants, there were substantial individual differences in the propensity to shift from goal-directed towards habitual behaviour, indicating that there is no overall effect of stress on the control of behaviour but that individuals differ in their sensitivity in these stress effects. This raises the important question of what makes individuals more or less vulnerable to the effects of stress on instrumental control (and cognition in general). Previous work suggested that genetic variants related to major stress response systems (i.e. noradrenergic and glucocorticoid activity) may play an important role in which multiple learning systems are engaged after stress (Wirz et al., 2017; Wirz et al., 2018). Understanding which further individual factors contribute to the interindividual propensity to shift from goal-directed to habitual behaviour needs to be addressed by future research. Furthermore, the fact that our neural data were not bimodal suggests that changes in neural representations may not translate directly into behavioural changes. The observed significant correlations between neural representation and performance in devalued trials show that there is a link between the behavioural and neural representation level. However, the correlations were obviously far below 1. Compared to the behavioural level which included only discrete yes-no responses, the neural data may have been much more sensitive and able to capture more fine-grained changes. The different sensitivity of behavioural and neural data is interesting in itself and points to another important question for future research: how do neural representation changes and behavioural changes relate to each other? Is there a particular threshold at which a change in representation triggers behavioural changes?

While we assume that the opposing effects of stress on neural outcome and response representations were due to the action of major stress response systems, there might have been other factors that have contributed to the present pattern of results, such as motivational factors or fatigue at the end of the task. Although we cannot completely rule out these alternatives, we consider them rather unlikely in light of our data. First, if these factors were a result of the mere amount of training, they should have also occurred in the control group which was not the case. Even if a specific interaction with the stress manipulation is assumed, it is important to note that reaction times remained fast in stressed participants until the end of training, and the response accuracy in valued trials or consumption trials remained very high. Furthermore, the observed specificity of the stress effects which occurred selectively in devalued trials cannot - in our view - be explained by unspecific influences, such as lack of motivation or fatigue.

In the present study, stress was induced before learning and outcome devaluation. Thus, stress could have affected the acquisition or the expression of instrumental behaviour, or both. While several previous studies demonstrated that acute stress (or the administration of stress hormones) before learning may shift instrumental behaviour from goal-directed to habitual control (Braun and Hauber, 2013; Dias-Ferreira et al., 2009; Gourley et al., 2012; Hartogsveld et al., 2020; Schwabe et al., 2010; Schwabe et al., 2011b; Schwabe et al., 2012; Schwabe and Wolf, 2009; Soares et al., 2012), there is evidence suggesting that stress before a test of behavioural expression may have a similar impact, that is, stress may induce habitual responding even when stress left acquisition unaffected (Schwabe et al., 2011b; Schwabe and Wolf, 2010). The latter finding, however, does not rule out additional effects of stress on acquisition, and indeed the impact of stress appeared to be more pronounced when the stress exposure took place before learning (Schwabe and Wolf, 2010). The present study did not aim to distinguish between stress effects on acquisition, or expression of goal-directed vs. habitual behaviour, but focussed on the impact of stress of the control of instrumental behaviour. Thus, our findings do not allow us to distinguish between stress effects on acquisition vs. expression of instrumental behaviour.

Based on the associative-cybernetic model (Dickinson and Balleine, 1993), it could be predicted that the obtained pattern of increased outcome and decreased response representations even leads to reduced responding for devalued outcomes across training in controls. This may be because individuals need to encode a representation of the response being performed in order to attain the outcome for a response to be goal-directed. We did not observe such a decrease, which may be attributed to the overall relatively low rate of responses for devalued outcomes in control participants.

Goal-directed action and habits are commonly considered to be two sides of the same coin. If a behaviour is more goal-directed, it is automatically less habitual (and vice versa). Obviously, behaviour cannot be fully goal-directed and habitual at the same time according to canonical operational definitions (Adams, 1982; Dickinson and Balleine, 1994). However, behaviour may not necessarily always be either fully goal-directed or habitual, and there may be different degrees to which behaviour is under goal-directed or habitual control. Given that the two modes of behavioural control are subserved by distinct neural circuits (Balleine and O’Doherty, 2010), it should be possible to separate goal-directed and habitual contributions to learning at a specific point in time. Classical behavioural paradigms involving discrete responses, however, cannot disentangle goal-directed and habitual components in a specific pattern of responding (e.g. insensitivity to outcome devaluation). Furthermore, tests of related cognitive functions, such as inhibitory control, provide only indirect evidence if any on the balance of goal-directed and habitual processes. Recently, a free-operant model was proposed that allows a behavioural dissociation of goal-directed and habitual contributions to behaviour (Perez and Dickinson, 2020). Here, we used here an MVPA-based decoding approach that focussed on neural representations that are a hallmark feature of goal-directed and habitual control. Using this novel approach, we show that acute stress reduces outcome representations and, at the same time, increases response representations in healthy humans. These were both directly linked to the stress-induced increase in habitual responding, suggesting that stress might exert opposite effects on goal-directed and habitual processing that manifest in the dominance of habitual behaviour under stress.

Data reported in this manuscript are available from the website: https://github.com/08122019/From-goal-directed-action-to-habit, copy archived at swh:1:rev:d52c6db7505b203310ec623db4fc07fede10eb9d.

1. 1. Adams CD
   2. Dickinson A

   (1981) Instrumental responding following reinforcer devaluation

   The Quarterly Journal of Experimental Psychology Section B 33:109–121.

   https://doi.org/10.1080/14640748108400816

   • Google Scholar
2. 1. Adams CD

   (1982) Variations in the sensitivity of instrumental responding to reinforcer devaluation

   The Quarterly Journal of Experimental Psychology Section B 34:77–98.

   https://doi.org/10.1080/14640748208400878

   • Google Scholar
3. 1. Adams TG
   2. Kelmendi B
   3. Brake CA
   4. Gruner P
   5. Badour CL
   6. Pittenger C

   (2018) The role of stress in the pathogenesis and maintenance of obsessive-compulsive disorder

   Chronic Stress 2:2470547018758043.

   https://doi.org/10.1177/2470547018758043

   • PubMed
   • Google Scholar
4. 1. Agresti A
   2. Caffo B

   (2000) Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures

   The American Statistician 54:280.

   https://doi.org/10.2307/2685779

   • Google Scholar
5. 1. Allen AP
   2. Kennedy PJ
   3. Cryan JF
   4. Dinan TG
   5. Clarke G

   (2014) Biological and psychological markers of stress in humans: focus on the Trier Social Stress Test

   Neuroscience and Biobehavioral Reviews 38:94–124.

   https://doi.org/10.1016/j.neubiorev.2013.11.005

   • PubMed
   • Google Scholar
6. 1. Bae GY
   2. Luck SJ

   (2018) Dissociable decoding of spatial attention and working memory from eeg oscillations and sustained potentials

   The Journal of Neuroscience 38:409–422.

   https://doi.org/10.1523/JNEUROSCI.2860-17.2017

   • PubMed
   • Google Scholar
7. 1. Balleine BW
   2. Dickinson A

   (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates

   Neuropharmacology 37:407–419.

   https://doi.org/10.1016/s0028-3908(98)00033-1

   • PubMed
   • Google Scholar
8. 1. Balleine BW
   2. O’Doherty JP

   (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

   Neuropsychopharmacology 35:48–69.

   https://doi.org/10.1038/npp.2009.131

   • PubMed
   • Google Scholar
9. Book

   1. Beck AT
   2. Steer RA
   3. Brown GK.

   (1996)

   Manual for the Beck Depression Inventory-II

   San Antonio: Psychological Corporation.

   • Google Scholar
10. 1. Bigdely-Shamlo N
    2. Mullen T
    3. Kothe C
    4. Su K-M
    5. Robbins KA

    (2015) The PREP pipeline: standardized preprocessing for large-scale EEG analysis

    Frontiers in Neuroinformatics 9:16.

    https://doi.org/10.3389/fninf.2015.00016

    • PubMed
    • Google Scholar
11. 1. Braun S
    2. Hauber W

    (2013) Acute stressor effects on goal-directed action in rats

    Learning & Memory 20:700–709.

    https://doi.org/10.1101/lm.032987.113

    • PubMed
    • Google Scholar
12. 1. Cairney SA
    2. Guttesen AÁV
    3. El Marj N
    4. Staresina BP

    (2018) Memory consolidation is linked to spindle-mediated information processing during sleep

    Current Biology 28:948–954.

    https://doi.org/10.1016/j.cub.2018.01.087

    • PubMed
    • Google Scholar
13. 1. Chaumon M
    2. Bishop DVM
    3. Busch NA

    (2015) A practical guide to the selection of independent components of the electroencephalogram for artifact correction

    Journal of Neuroscience Methods 250:47–63.

    https://doi.org/10.1016/j.jneumeth.2015.02.025

    • PubMed
    • Google Scholar
14. 1. Corbit LH
    2. Balleine BW

    (2003) The role of prelimbic cortex in instrumental conditioning

    Behavioural Brain Research 146:145–157.

    https://doi.org/10.1016/j.bbr.2003.09.023

    • PubMed
    • Google Scholar
15. 1. de Kloet ER
    2. Karst H
    3. Joëls M

    (2008) Corticosteroid hormones in the central stress response: quick-and-slow

    Frontiers in Neuroendocrinology 29:268–272.

    https://doi.org/10.1016/j.yfrne.2007.10.002

    • PubMed
    • Google Scholar
16. 1. de Quervain DJ
    2. Roozendaal B
    3. McGaugh JL

    (1998) Stress and glucocorticoids impair retrieval of long-term spatial memory

    Nature 394:787–790.

    https://doi.org/10.1038/29542

    • PubMed
    • Google Scholar
17. 1. de Wit S
    2. Kindt M
    3. Knot SL
    4. Verhoeven AAC
    5. Robbins TW
    6. Gasull-Camos J
    7. Evans M
    8. Mirza H
    9. Gillan CM

    (2018) Shifting the balance between goals and habits: Five failures in experimental habit induction

    Journal of Experimental Psychology. General 147:1043–1065.

    https://doi.org/10.1037/xge0000402

    • PubMed
    • Google Scholar
18. 1. Delorme A
    2. Makeig S

    (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis

    Journal of Neuroscience Methods 134:9–21.

    https://doi.org/10.1016/j.jneumeth.2003.10.009

    • PubMed
    • Google Scholar
19. 1. Dias-Ferreira E
    2. Sousa JC
    3. Melo I
    4. Morgado P
    5. Mesquita AR
    6. Cerqueira JJ
    7. Costa RM
    8. Sousa N

    (2009) Chronic stress causes frontostriatal reorganization and affects decision-making

    Science 325:621–625.

    https://doi.org/10.1126/science.1171203

    • PubMed
    • Google Scholar
20. 1. Dickinson A
    2. Balleine B

    (1993)

    Spatial Representation: Problems in Philosophy and Psychology

    Actions and responses: The dual psychology of behaviour, Spatial Representation: Problems in Philosophy and Psychology, Blackwell Publishing.

    • Google Scholar
21. 1. Dickinson A
    2. Balleine B

    (1994) Motivational control of goal-directed action

    Animal Learning & Behavior 22:1–18.

    https://doi.org/10.3758/BF03199951

    • Google Scholar
22. 1. Dickinson A
    2. Balleine B
    3. Watt A
    4. Gonzalez F
    5. Boakes RA

    (1995) Motivational control after extended instrumental training

    Animal Learning & Behavior 23:197–206.

    https://doi.org/10.3758/BF03199935

    • Google Scholar
23. 1. Domes G
    2. Zimmer P

    (2019) Acute stress enhances the sensitivity for facial emotions: a signal detection approach

    Stress 22:455–460.

    https://doi.org/10.1080/10253890.2019.1593366

    • PubMed
    • Google Scholar
24. 1. Everitt BJ
    2. Robbins TW

    (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion

    Nature Neuroscience 8:1481–1489.

    https://doi.org/10.1038/nn1579

    • PubMed
    • Google Scholar
25. 1. Gagnon SA
    2. Wagner AD

    (2016) Acute stress and episodic memory retrieval: neurobiological mechanisms and behavioral consequences

    Annals of the New York Academy of Sciences 1369:55–75.

    https://doi.org/10.1111/nyas.12996

    • PubMed
    • Google Scholar
26. 1. Goeders NE

    (2004) Stress, motivation, and drug addiction

    Current Directions in Psychological Science 13:33–35.

    https://doi.org/10.1111/j.0963-7214.2004.01301009.x

    • Google Scholar
27. 1. Gourley SL
    2. Swanson AM
    3. Jacobs AM
    4. Howell JL
    5. Mo M
    6. Dileone RJ
    7. Koleske AJ
    8. Taylor JR

    (2012) Action control is mediated by prefrontal BDNF and glucocorticoid receptor binding

    PNAS 109:20714–20719.

    https://doi.org/10.1073/pnas.1208342109

    • PubMed
    • Google Scholar
28. 1. Griffiths KR
    2. Morris RW
    3. Balleine BW

    (2014) Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

    Frontiers in Systems Neuroscience 8:101.

    https://doi.org/10.3389/fnsys.2014.00101

    • PubMed
    • Google Scholar
29. 1. Hartogsveld B
    2. van Ruitenbeek P
    3. Quaedflieg CWEM
    4. Smeets T

    (2020) Balancing between goal-directed and habitual responding following acute stress

    Experimental Psychology 67:99–111.

    https://doi.org/10.1027/1618-3169/a000485

    • PubMed
    • Google Scholar
30. 1. Hickey C
    2. Chelazzi L
    3. Theeuwes J

    (2010) Reward changes salience in human vision via the anterior cingulate

    The Journal of Neuroscience 30:11096–11103.

    https://doi.org/10.1523/JNEUROSCI.1026-10.2010

    • PubMed
    • Google Scholar
31. Book

    1. Hoijtink H.

    (2012)

    Informative hypotheses: Theory and practice for behavioral and social scientists

    Boca Raton: CRC.

    • Google Scholar
32. 1. Isik L
    2. Meyers EM
    3. Leibo JZ
    4. Poggio T

    (2014) The dynamics of invariant object recognition in the human visual system

    Journal of Neurophysiology 111:91–102.

    https://doi.org/10.1152/jn.00394.2013

    • PubMed
    • Google Scholar
33. 1. Joëls M
    2. Baram TZ

    (2009) The neuro-symphony of stress

    Nature Reviews. Neuroscience 10:459–466.

    https://doi.org/10.1038/nrn2632

    • PubMed
    • Google Scholar
34. 1. Joëls M
    2. Sarabdjitsingh RA
    3. Karst H

    (2012) Unraveling the time domains of corticosteroid hormone influences on brain activity: rapid, slow, and chronic modes

    Pharmacological Reviews 64:901–938.

    https://doi.org/10.1124/pr.112.005892

    • PubMed
    • Google Scholar
35. 1. Kass RE
    2. Raftery AE

    (1995) Bayes Factors

    Journal of the American Statistical Association 90:773–795.

    https://doi.org/10.1080/01621459.1995.10476572

    • Google Scholar
36. 1. Kirschbaum C
    2. Pirke KM
    3. Hellhammer DH

    (1993) The ’Trier Social Stress Test’--a tool for investigating psychobiological stress responses in a laboratory setting

    Neuropsychobiology 28:76–81.

    https://doi.org/10.1159/000119004

    • PubMed
    • Google Scholar
37. 1. Luque D
    2. Beesley T
    3. Morris RW
    4. Jack BN
    5. Griffiths O
    6. Whitford TJ
    7. Le Pelley ME

    (2017) Goal-directed and habit-like modulations of stimulus processing during reinforcement learning

    The Journal of Neuroscience 37:3009–3017.

    https://doi.org/10.1523/JNEUROSCI.3205-16.2017

    • PubMed
    • Google Scholar
38. 1. MacLean MH
    2. Giesbrecht B

    (2015) Neural evidence reveals the rapid effects of reward history on selective attention

    Brain Research 1606:86–94.

    https://doi.org/10.1016/j.brainres.2015.02.016

    • PubMed
    • Google Scholar
39. 1. McNamee D
    2. Liljeholm M
    3. Zika O
    4. O’Doherty JP

    (2015) Characterizing the associative content of brain structures involved in habitual and goal-directed actions in humans: A multivariate FMRI study

    The Journal of Neuroscience 35:3764–3771.

    https://doi.org/10.1523/JNEUROSCI.4677-14.2015

    • PubMed
    • Google Scholar
40. 1. Mognon A
    2. Jovicich J
    3. Bruzzone L
    4. Buiatti M

    (2011) ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features

    Psychophysiology 48:229–240.

    https://doi.org/10.1111/j.1469-8986.2010.01061.x

    • PubMed
    • Google Scholar
41. 1. Müller-Putz G
    2. Scherer R
    3. Brunner C
    4. Leeb R
    5. Pfurtscheller G

    (2008)

    Better than random? A closer look on BCI results

    International Journal of Bioelectromagnetism 10:52–55.

    • Google Scholar
42. 1. Murphy B
    2. Poesio M
    3. Bovolo F
    4. Bruzzone L
    5. Dalponte M
    6. Lakany H

    (2011) EEG decoding of semantic category reveals distributed representations for single concepts

    Brain and Language 117:12–22.

    https://doi.org/10.1016/j.bandl.2010.09.013

    • PubMed
    • Google Scholar
43. 1. Noh E
    2. Herzmann G
    3. Curran T
    4. de Sa VR

    (2014) Using single-trial EEG to predict and analyze subsequent memory

    NeuroImage 84:712–723.

    https://doi.org/10.1016/j.neuroimage.2013.09.028

    • PubMed
    • Google Scholar
44. 1. Nuzzo RL

    (2017) An introduction to bayesian data analysis for correlations

    PM & R 9:1278–1282.

    https://doi.org/10.1016/j.pmrj.2017.11.003

    • PubMed
    • Google Scholar
45. 1. Oostenveld R
    2. Fries P
    3. Maris E
    4. Schoffelen JM

    (2011) FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data

    Computational Intelligence and Neuroscience 2011:156869.

    https://doi.org/10.1155/2011/156869

    • PubMed
    • Google Scholar
46. 1. Ostlund SB
    2. Balleine BW

    (2005) Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning

    The Journal of Neuroscience 25:7763–7770.

    https://doi.org/10.1523/JNEUROSCI.1921-05.2005

    • PubMed
    • Google Scholar
47. 1. Packard MG

    (1999) Glutamate infused posttraining into the hippocampus or caudate-putamen differentially strengthens place and response learning

    PNAS 96:12881–12886.

    https://doi.org/10.1073/pnas.96.22.12881

    • PubMed
    • Google Scholar
48. 1. Perez OD
    2. Dickinson A

    (2020) A theory of actions and habits: The interaction of rate correlation and contiguity systems in free-operant behavior

    Psychological Review 127:945–971.

    https://doi.org/10.1037/rev0000201

    • PubMed
    • Google Scholar
49. 1. Robbins TW
    2. Gillan CM
    3. Smith DG
    4. de Wit S
    5. Ersche KD

    (2012) Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry

    Trends in Cognitive Sciences 16:81–91.

    https://doi.org/10.1016/j.tics.2011.11.009

    • PubMed
    • Google Scholar
50. 1. Roozendaal B

    (2002) Stress and memory: opposing effects of glucocorticoids on memory consolidation and memory retrieval

    Neurobiology of Learning and Memory 78:578–595.

    https://doi.org/10.1006/nlme.2002.4080

    • PubMed
    • Google Scholar
51. 1. Schulz P
    2. Schlotz W

    (1999) The Trier Inventory for the Assessment of Chronic Stress (TICS): scale construction, statistical testing, and validation of the scale work overload

    Diagnostica 45:8–19.

    https://doi.org/10.1026//0012-1924.45.1.8

    • Google Scholar
52. 1. Schwabe L
    2. Oitzl MS
    3. Richter S
    4. Schächinger H

    (2009) Modulation of spatial and stimulus-response learning strategies by exogenous cortisol in healthy young women

    Psychoneuroendocrinology 34:358–366.

    https://doi.org/10.1016/j.psyneuen.2008.09.018

    • PubMed
    • Google Scholar
53. 1. Schwabe L
    2. Wolf OT

    (2009) Stress prompts habit behavior in humans

    The Journal of Neuroscience 29:7191–7198.

    https://doi.org/10.1523/JNEUROSCI.0979-09.2009

    • PubMed
    • Google Scholar
54. 1. Schwabe L
    2. Tegenthoff M
    3. Höffken O
    4. Wolf OT

    (2010) Concurrent glucocorticoid and noradrenergic activity shifts instrumental behavior from goal-directed to habitual control

    The Journal of Neuroscience 30:8190–8196.

    https://doi.org/10.1523/JNEUROSCI.0734-10.2010

    • PubMed
    • Google Scholar
55. 1. Schwabe L
    2. Wolf OT

    (2010) Socially evaluated cold pressor stress after instrumental learning favors habits over goal-directed action

    Psychoneuroendocrinology 35:977–986.

    https://doi.org/10.1016/j.psyneuen.2009.12.010

    • PubMed
    • Google Scholar
56. 1. Schwabe L
    2. Dickinson A
    3. Wolf OT

    (2011a) Stress, habits, and drug addiction: a psychoneuroendocrinological perspective

    Experimental and Clinical Psychopharmacology 19:53–63.

    https://doi.org/10.1037/a0022212

    • PubMed
    • Google Scholar
57. 1. Schwabe L
    2. Höffken O
    3. Tegenthoff M
    4. Wolf OT

    (2011b) Preventing the stress-induced shift from goal-directed to habit action with a β-adrenergic antagonist

    The Journal of Neuroscience 31:17317–17325.

    https://doi.org/10.1523/JNEUROSCI.3304-11.2011

    • PubMed
    • Google Scholar
58. 1. Schwabe L
    2. Tegenthoff M
    3. Höffken O
    4. Wolf OT

    (2012) Simultaneous glucocorticoid and noradrenergic activity disrupts the neural basis of goal-directed action in the human brain

    The Journal of Neuroscience 32:10146–10155.

    https://doi.org/10.1523/JNEUROSCI.1304-12.2012

    • PubMed
    • Google Scholar
59. 1. Schwabe L
    2. Wolf OT

    (2012) Stress modulates the engagement of multiple memory systems in classification learning

    The Journal of Neuroscience 32:11042–11049.

    https://doi.org/10.1523/JNEUROSCI.1484-12.2012

    • PubMed
    • Google Scholar
60. 1. Schwabe L
    2. Wolf OT

    (2013) Stress and multiple memory systems: from “thinking” to “doing.”

    Trends in Cognitive Sciences 17:60–68.

    https://doi.org/10.1016/j.tics.2012.12.001

    • PubMed
    • Google Scholar
61. 1. Seehagen S
    2. Schneider S
    3. Rudolph J
    4. Ernst S
    5. Zmyj N

    (2015) Stress impairs cognitive flexibility in infants

    PNAS 112:12882–12886.

    https://doi.org/10.1073/pnas.1508345112

    • PubMed
    • Google Scholar
62. 1. Siller-Pérez C
    2. Serafín N
    3. Prado-Alcalá RA
    4. Roozendaal B
    5. Quirarte GL

    (2017) Glucocorticoid administration into the dorsolateral but not dorsomedial striatum accelerates the shift from a spatial toward procedural memory

    Neurobiology of Learning and Memory 141:124–133.

    https://doi.org/10.1016/j.nlm.2017.03.020

    • PubMed
    • Google Scholar
63. 1. Smeets T
    2. Quaedflieg C

    (2019) How stress and cognitive fatigue change the balancing between goal-directed and habitual behavior

    Psychoneuroendocrinology 107:71.

    https://doi.org/10.1016/j.psyneuen.2019.07.204

    • Google Scholar
64. 1. Smeets T
    2. van Ruitenbeek P
    3. Hartogsveld B
    4. Quaedflieg CWEM

    (2019) Stress-induced reliance on habitual behavior is moderated by cortisol reactivity

    Brain and Cognition 133:60–71.

    https://doi.org/10.1016/j.bandc.2018.05.005

    • PubMed
    • Google Scholar
65. 1. Soares JM
    2. Sampaio A
    3. Ferreira LM
    4. Santos NC
    5. Marques F
    6. Palha JA
    7. Cerqueira JJ
    8. Sousa N

    (2012) Stress-induced changes in human decision-making are reversible

    Translational Psychiatry 2:e131.

    https://doi.org/10.1038/tp.2012.59

    • PubMed
    • Google Scholar
66. Book

    1. Spielberger CD.

    (1983)

    Manual for the State-Trait Anxiety Inventory

    Palo Alto, CA: Consulting Psychologists Pres.

    • Google Scholar
67. 1. Steyer R
    2. Schwenkmezger P
    3. Notz P
    4. Eid M

    (1994)

    Testtheoretische analysen des mehrdimensionalen befindlichkeitsfragebogen (mdbf)

    Diagnostica 40:320–328.

    • Google Scholar
68. 1. Taghizadeh-Sarabi M
    2. Daliri MR
    3. Niksirat KS

    (2015) Decoding objects of basic categories from electroencephalographic signals using wavelet transform and support vector machines

    Brain Topography 28:33–46.

    https://doi.org/10.1007/s10548-014-0371-9

    • PubMed
    • Google Scholar
69. 1. Tanaka SC
    2. Balleine BW
    3. O’Doherty JP

    (2008) Calculating consequences: brain systems that encode the causal effects of actions

    The Journal of Neuroscience 28:6750–6755.

    https://doi.org/10.1523/JNEUROSCI.1808-08.2008

    • PubMed
    • Google Scholar
70. 1. Treder MS

    (2020) Mvpa-light: A classification and regression toolbox for multi-dimensional data

    Frontiers in Neuroscience 14:289.

    https://doi.org/10.3389/fnins.2020.00289

    • PubMed
    • Google Scholar
71. 1. Tricomi E
    2. Balleine BW
    3. O’Doherty JP

    (2009) A specific role for posterior dorsolateral striatum in human habit learning

    The European Journal of Neuroscience 29:2225–2232.

    https://doi.org/10.1111/j.1460-9568.2009.06796.x

    • PubMed
    • Google Scholar
72. 1. Valentin VV
    2. Dickinson A
    3. O’Doherty JP

    (2007) Determining the neural substrates of goal-directed learning in the human brain

    The Journal of Neuroscience 27:4019–4026.

    https://doi.org/10.1523/JNEUROSCI.0564-07.2007

    • PubMed
    • Google Scholar
73. 1. Voon V
    2. Derbyshire K
    3. Rück C
    4. Irvine MA
    5. Worbe Y
    6. Enander J
    7. Schreiber LRN
    8. Gillan C
    9. Fineberg NA
    10. Sahakian BJ
    11. Robbins TW
    12. Harrison NA
    13. Wood J
    14. Daw ND
    15. Dayan P
    16. Grant JE
    17. Bullmore ET

    (2015) Disorders of compulsivity: a common bias towards learning habits

    Molecular Psychiatry 20:345–352.

    https://doi.org/10.1038/mp.2014.44

    • PubMed
    • Google Scholar
74. 1. Wirz L
    2. Wacker J
    3. Felten A
    4. Reuter M
    5. Schwabe L

    (2017) A deletion variant of the α2b-adrenoceptor modulates the stress-induced shift from “cognitive” to “habit” memory

    The Journal of Neuroscience 37:2149–2160.

    https://doi.org/10.1523/JNEUROSCI.3507-16.2017

    • PubMed
    • Google Scholar
75. 1. Wirz L
    2. Bogdanov M
    3. Schwabe L

    (2018) Habits under stress: mechanistic insights across different types of learning

    Current Opinion in Behavioral Sciences 20:9–16.

    https://doi.org/10.1016/j.cobeha.2017.08.009

    • Google Scholar
76. 1. Yin HH
    2. Knowlton BJ
    3. Balleine BW

    (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

    The European Journal of Neuroscience 19:181–189.

    https://doi.org/10.1111/j.1460-9568.2004.03095.x

    • PubMed
    • Google Scholar
77. 1. Yin HH
    2. Knowlton BJ
    3. Balleine BW

    (2006) Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning

    Behavioural Brain Research 166:189–196.

    https://doi.org/10.1016/j.bbr.2005.07.012

    • PubMed
    • Google Scholar

Author details

1. Jacqueline Katharina Meier

   Department of Cognitive Psychology, Universität Hamburg, Hamburg, Germany

   Contribution

   Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing - original draft

   Competing interests

   No competing interests declared

2. Bernhard P Staresina

   Department of Experimental Psychology, and Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, United Kingdom

   Contribution

   Conceptualization, Funding acquisition, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0558-9745

3. Lars Schwabe

   Department of Cognitive Psychology, Universität Hamburg, Hamburg, Germany

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Methodology, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   lars.schwabe@uni-hamburg.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4429-4373

• 1,326

  views

• 296

  downloads

• 11

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.