The perception of biological motion (BM), the movements of living creatures, is a fundamental ability of the human visual system. Extensive evidence shows that humans can readily perceive BM from a visual display depicting just a handful of light dots attached to the head and major joints of a moving person (Blake and Shiffrar, 2007). Nevertheless, in real life, BM perception often occurs in multisensory contexts. For instance, one may simultaneously hear footstep sounds while seeing others walking. The integration of these visual and auditory BM cues facilitates the detection, discrimination, and attentional processing of BM (Mendonça et al., 2011; Shen et al., 2023a; Thomas and Shiffrar, 2013; van der Zwan et al., 2009). Notably, such benefits are diminished when the visual BM is deprived of characteristic kinematic cues but not low-level motion attributes (Brooks et al., 2007; Shen et al., 2023a; Thomas and Shiffrar, 2010), and the temporal windows of perceptual audiovisual synchrony are different between BM and non-BM stimuli (Arrighi et al., 2006; Saygin et al., 2008), highlighting the specificity of audiovisual BM processing. This specificity may relate to the evolutionary significance of BM and its pivotal role in social situations. In particular, integrating multisensory BM cues is foundational for perceiving and attending to other people and developing further social interaction. Such ability is usually compromised in people with social deficits, such as individuals with autism spectrum disorder (ASD) (Falck-Ytter et al., 2013; Feldman et al., 2018). These findings underline the unique contribution of multisensory BM processing to human perception and social cognition. However, despite the behavioral evidence, the neural coding of audiovisual BM cues and its possible link with individuals’ social cognitive capability remains largely unexplored.

An intrinsic property of human movements (such as walking and running) is that they are rhythmic and accompanied by frequency-congruent sounds. The audiovisual integration (AVI) of such rhythmic stimuli may involve a process whereby brain activity aligns with and tracks external rhythms, revealed by increased power or phase coherence of neural oscillations at corresponding frequencies (Bauer et al., 2020; Ding et al., 2016; Obleser and Kayser, 2019). Studies based on simple or discrete stimuli show that temporal congruency in auditory and visual rhythms significantly enhances the cortical tracking of rhythmic stimulations in both modalities (Covic et al., 2017; Keitel and Müller, 2016; Nozaradan et al., 2012b). Unlike these stimuli, BM conveys complex hierarchical rhythmic structures corresponding to integration windows at multiple temporal scales. For example, the human locomotion movement has a narrower integration window consisting of each step (i.e. step cycle) and a broader integration window incorporating the opponent motion of the two feet (i.e. gait cycle). A recent study suggests that neural tracking of these nested kinematic structures contributes to the spatiotemporal integration of visual BM cues in different manners (Shen et al., 2023b). However, it remains open whether and how the cortical tracking of hierarchical rhythmic structures underpins the AVI of BM information.

To tackle this issue, we recorded electroencephalogram (EEG) signals from participants who viewed rhythmic point-light walkers or/and listened to the corresponding footstep sounds under visual (V), auditory (A), and audiovisual (AV) conditions in Experiments 1a and 1b (Figure 1). An enhanced cortical tracking effect in the AV condition compared to each unisensory condition will indicate significant multisensory gains. Moreover, we adopted an additive model to classify multisensory integration based on the AV vs. A+V comparison. This model assumes independence between inputs from each sensory modality and distinguishes among sub-additive (AV<A +V), additive (AV = A +V), and super-additive (AV>A +V) response modes (see a review by Stevenson et al., 2014). The additive mode represents a linear combination between two modalities. In contrast, the super-additive and sub-additive modes indicate non-linear interaction processing, either with potentiated neural activation to facilitate the perception or detection of near-threshold signals (super-additive) or a deactivation mechanism to minimize the processing of redundant information cross-modally (sub-additive) (Laurienti et al., 2005; Metzger et al., 2020; Stanford et al., 2005; Wright et al., 2003). Distinguishing among these integration modes may help elucidate the neural mechanism underlying AVI in specific contexts.

Figure 1

Download asset Open asset

Experiment 2 examined to what extent the AVI effect was specific to the multisensory processing of BM by using non-BM (inverted visual stimuli) as a control. Inversion disrupts the unique, gravity-compatible kinematic features of BM but not the rhythmic signals generated by low-level motion cues (Ma et al., 2022; Shen et al., 2023b; Simion et al., 2008; Troje and Westhoff, 2006; Wang et al., 2022), thus is expected to interfere with the BM-specific neural processing. Participants perceived visual BM stimuli accompanied by temporally congruent or incongruent BM sounds. Comparing the congruency effect in neural responses between the upright and inverted conditions allowed us to verify whether the AVI of BM involves a mechanism distinct from that underlies the AVI of non-BM. In addition, we further explored the possible linkage between the BM-specific neural tracking effect and observers’ autistic traits. Previous behavioral studies found reduced orienting to audiovisually synchronized BM stimuli in ASD (Falck-Ytter et al., 2018). Since individuals with varying social cognitive abilities lie on a continuum extending from clinical to nonclinical populations with different levels of autistic traits, we investigated whether cortical tracking of audiovisual BM correlates with individuals’ autistic traits, as measured by the Autism-Spectrum Quotient (AQ) (Baron-Cohen et al., 2001).

In all experiments, 17–23% of the trials were randomly selected as catch trials, in which the color of the walker changed one or two times throughout the trial, and there was no color change in other trials. Participants were required to detect the color change of visual stimuli (zero to two times during one trial) to maintain attention. Behavioral analysis on all trials showed that their performances for the task were generally high and equally well in all conditions of Experiment 1a (mean accuracy>98%; F (2, 46)=0.814, p=0.450, ƞ_p²=20.034), Experiment 1b (mean accuracy>98%; F (2, 46)=0.615, p=0.545, ƞ_p²=20.026), and Experiment 2 (mean accuracy>98%; F (3, 69)=0.493, p=0.688, ƞ_p²=20.021), indicating comparable attention state across conditions. The catch trials were excluded from the following EEG analysis.

Cortical tracking of rhythmic structures in audiovisual BM reveals AVI

Experiment 1a

In Experiment 1a, we examined the cortical tracking of rhythmic BM information under V, A, and AV conditions (Figure 1c). We were interested in two critical rhythmic structures in the walking motion sequence, i.e., the gait cycle and the step cycle (Figure 1a and b). During walking, each step of the left or right foot occurs alternatively to form a step cycle, and the antiphase oscillations of limbs during two steps characterize a gait cycle (Shen et al., 2023b). In Experiment 1a, the frequency of a full gait cycle is 1Hz, and the step-cycle frequency is 2Hz. The strength of the cortical tracking effect was quantified by the amplitude peaks emerging from the EEG spectra at these frequencies.

As shown in the grand average amplitude spectra (Figure 2a), both the responses in three conditions showed clear peaks at step-cycle frequency (2Hz; V: t (23)=6.963, p<0.001; A: t (23)=6.073, p<0.001; AV: t (23)=7.054, p<0.001; FDR corrected). In contrast, at gait-cycle frequency (1Hz), only the response to AV stimulation showed significant peaks (V: t (23)=–2.072, p=0.975; A: t (23)=–0.054, p=0.521; AV: t (23)=4.059, p<0.001; FDR corrected). Besides, we also observed significant peaks at 4Hz in all three conditions (ps<0.001, FDR corrected), which showed a similar audiovisual integration mode as 2Hz (see more details in Appendix and Figure 2—figure supplement 1).

Figure 2 with 1 supplement see all

Download asset Open asset

Furthermore, we directly compared the cortical tracking effects between different conditions via a two-tailed paired t-test. At both 1Hz (Figure 2b) and 2Hz (Figure 2c), the amplitude in the AV condition was greater than that in the V condition (1Hz: t (23)=4.664, p<0.001, Cohen’s d=0.952; 2Hz: t (23)=5.132, p<0.001, Cohen’s d=1.048) and the A condition (1Hz: t (23)=2.391, p=0.025, Cohen’s d=0.488; 2Hz: t (23)=3.808, p<0.001, Cohen’s d=0.777), respectively, suggesting multisensory gains. More importantly, at 1Hz, the amplitude in the AV condition was significantly larger than the algebraic sum of those in the A and V conditions (t (23)=3.028, p=0.006, Cohen’s d=0.618), indicating a super-additive audiovisual integration effect. While at 2Hz, the amplitude in the AV condition was comparable to the unisensory sum (t (23)=–0.623, p=0.539, Cohen’s d=–0.127), indicating additive audiovisual integration.

Experiment 1b

To further test whether such cortical tracking effect can apply to stimuli with a different speed, Experiment 1b altered the frequencies of the gait cycle and the corresponding step cycle to 0.83Hz and 1.67Hz while adopting the same paradigm as Experiment 1a. Consistent with Experiment 1a, the frequency-domain analysis revealed significant cortical tracking of the audiovisual stimuli at the new speeds. As shown in Figure 2d, both the responses to V, A, and AV stimuli showed clear peaks at step-cycle frequency (1.67Hz; V: t (23)=3.473, p=0.001; A: t (23)=9.194, p<0.001; AV: t (23)=8.756, p<0.001; FDR corrected) and its harmonics (3.33Hz, ps<0.001, FDR corrected). In contrast, at gait-cycle frequency (0.83Hz), only the response to AV stimuli showed significant peaks (V: t (23)=–1.125, p=0.846; A: t (23)=–2.449, p=0.989; AV: t (23)=3.052, p=0.003; FDR corrected).

At both 0.83Hz (Figure 2e) and 1.67Hz (Figure 2f), the amplitude in the AV condition was stronger or marginally stronger than that in the V condition (0.83Hz: t (23)=2.665, p=0.014, Cohen’s d=0.544; 1.67Hz: t (23)=6.380, p<0.001, Cohen’s d=1.302) and the A condition (0.83Hz: t (23)=3.625, p<0.001, Cohen’s d=0.740; 1.67Hz: t (23)=1.752, p=0.093, Cohen’s d=0.358), respectively, suggesting multisensory gains. More importantly, at 0.83Hz, the amplitude in the AV condition was significantly larger than the sum of those in the A and V conditions (t (23)=3.240, p=0.004, Cohen’s d=0.661), indicating a super-additive audiovisual integration effect. By contrast, at 1.67Hz, the amplitude in the AV condition was comparable to the unisensory sum (t (23)=–0.735, p=0.470, Cohen’s d=–0.150), indicating linear audiovisual integration. Significant peaks were also observed at 3.33Hz in all three conditions (ps<0.001, FDR corrected), which showed similar audiovisual integration mode as 1.67Hz (see more details in Appendix and Figure 2—figure supplement 1).

In summary, results from Experiments 1a and 1b consistently showed that the cortical tracking of the audiovisual signals at different temporal scales exhibit distinct audiovisual integration modes, i.e., the super-additive effect at gait-cycle frequency and the additive effect at step-cycle frequency, indicating that the cortical tracking effects at the two temporal scales might be driven by functionally dissociable mechanisms.

Cortical tracking of higher-order rhythmic structure contributes to the AVI of BM

To further explore whether and how the cortical tracking of rhythmic structures contributes to the specialized audiovisual processing of BM, we adopted both upright and inverted BM stimuli in Experiment 2. The task and the frequencies of visual stimuli in Experiment 2 were same as Experiment 1a. Specifically, participants were required to perform the change detection task when perceiving upright and inverted visual BM sequences (1Hz for gait-cycle frequency and 2Hz for step-cycle frequency) accompanied by frequency congruent (1Hz) or incongruent (0.6Hz and 1.4Hz) footstep sounds (Figure 1c). The audiovisual congruency effect, characterized by stronger neural responses in the audiovisual congruent condition compared with the incongruent condition, can be taken as an index of AVI (Fleming et al., 2020; Maddox et al., 2015; Wuerger et al., 2012a). A stronger congruency effect in the upright condition relative to the inverted condition characterizes an AVI process specific to BM information.

We contrasted the congruency effect between the upright and inverted conditions to search for clusters showing a significant difference, which equaled identifying an interaction effect, using a cluster-based permutation test over all electrodes (n=1000, alpha = 0.05; see Materials and methods). At 1Hz, the congruency effect in the upright condition was significantly stronger than that in the inverted condition in a cluster at the right hemisphere (Figure 3a, lower panel, p=0.029; C2, CPz, CP2, CP4, CP6, Pz, P2, P4, P6). Then, we averaged the amplitude of electrodes within the significant cluster and performed two-tailed paired t-tests to examine whether the congruency effect was significant in the upright and the inverted conditions, respectively. Results showed that (Figure 3b) audiovisual congruency enhanced the oscillatory amplitude only for upright BM (t (23)=4.632, p<0.001, Cohen’s d=0.945) but not when visual BM was inverted (t (23)=0.480, p=0.635, Cohen’s d=0.098). Together, these findings suggest that cortical tracking of the high-order gait cycles involves a domain-specific process exclusively engaged in the AVI of BM.

Figure 3 with 2 supplements see all

Download asset Open asset

In contrast, at 2Hz, no cluster showed a significantly different congruency effect between the upright and inverted conditions (Figure 3d). We then conducted further analysis based on the electrodes yielded by 1Hz as marked in Figure 3a. Results showed that both upright and inverted stimuli induced a significant congruency effect at 2Hz (Figure 3e; upright: t (23)=3.096, p=0.005, Cohen’s d=0.632; inverted: t (23)=2.672, p=0.014, Cohen’s d=0.545). These findings suggest that neural tracking of the lower-order step cycles is associated with a domain-general AVI process mostly driven by temporal correspondence in physical stimuli.

The current study investigated the neural implementation for the AVI of human BM information. We found that, even under a motion-irrelevant color detection task, observers’ neural activity tracked the temporally corresponding audiovisual BM signals at the frequencies of two rhythmic structures, i.e., the higher-order structure of gait cycle at a larger integration window and the basic-level structure of step cycle at a smaller integration window. The strength of these cortical tracking effects was enhanced under the audiovisual condition than in the visual-only or auditory-only condition, indicating multisensory gains. More crucially, although the cortical tracking of both gait-cycle and step-cycle gain benefits from multisensory correspondence, the mechanisms underlying these two processes appear to be different. At step-cycle frequency, the cortical tracking effect in the AV condition equaled the additive sum of the unisensory conditions. Such linear integration may result from concurrent, independent processing of unisensory inputs without additional interaction of them (Stein et al., 2009). In contrast, at gait-cycle frequency, the congruent audiovisual signals led to a super-additive multisensory enhancement over the linear combination of auditory and visual conditions (AV>A +V), despite that there was no evident cortical tracking effect in the visual condition, different from previous findings obtained with a motion-relevant change detection task (Shen et al., 2023b). This super-additive multisensory enhancement may bring about decreased thresholds of detection and identification (Stanford et al., 2005), allowing people to achieve a more clear and stable perception of the external environment and detect weak stimulus changes in time and respond adaptively.

Furthermore, results from Experiment 2 demonstrated that the cortical tracking of rhythmic structure corresponding to the gait cycle rather than the step cycle underlies the specialized processing of audiovisual BM information. In particular, the AVI effect at step-cycle frequency was significant for both upright and inverted BM signals and comparable between the two conditions, while the AVI effect at gait-cycle frequency was only significant in the upright condition and was greater than that in the inverted condition. The inversion effect has long been regarded as a marker of the specificity of BM processing in numerous behavioral and neuroimaging studies (Grossman and Blake, 2001; Ma et al., 2022; Shen et al., 2023b; Simion et al., 2008; Troje and Westhoff, 2006; Vallortigara and Regolin, 2006; Wang et al., 2014; Wang and Jiang, 2012; Wang et al., 2022). Our current findings of the inversion effect in the cortical tracking of audiovisual BM at the gait-cycle frequency suggest that the neural encoding of the higher-order rhythmic structure reflects the AVI of BM and contributes to the specialized processing of BM information. In contrast, the cortical tracking of the step cycle may reflect the integration of basic motion signals and corresponding sounds. Together, these results reveal that the neural tracking of rhythmic structures at different temporal scales plays distinct roles in the AVI of BM, which may result from the interplay of stimulus-driven and domain-specific mechanisms.

Besides the temporal dynamics of neural activity revealed by the cortical tracking process, we found that the BM-specific AVI effect was associated with neural activity in the right temporoparietal electrodes. This finding likely relates to the activation of the right posterior superior temporal sulcus (pSTS), a region responding to both auditory and visual BM information and being causally involved in BM perception (Bidet-Caulet et al., 2005; Grossman et al., 2005; Wang et al., 2022). While previous fMRI studies have observed STS activation when processing spatial or semantic correspondence between audiovisual BM (Meyer et al., 2011; Wuerger et al., 2012b), whether this region also engages in the audiovisual processing of BM signals based on temporal correspondence remains unknown. The current study provides preliminary evidence for such a possibility, inviting future research to localize the exact source of the multisensory integration processes based on imaging data with high spatial and temporal resolutions, such as MEG.

Cortical tracking of external rhythms is also described as cortical entrainment in a broad sense (Ding et al., 2016; Obleser and Kayser, 2019). Controversy remains regarding the involvements of endogenous neural oscillations and stimulus-evoked responses in these processes (Duecker et al., 2024), as it is challenging to fully dissociate these components due to their intricate interplay (Herrmann et al., 2016; Hosseinian et al., 2021). Despite the complexity of the neuronal mechanisms, previous research suggests that cortical tracking or entrainment plays a role in the multisensory processing of simple or discrete rhythmic signals (Covic et al., 2017; Keitel and Müller, 2016; Nozaradan et al., 2012b). These findings may partially explain the non-selective AVI effect at the step cycle in the current study. However, we found that the cortical tracking of the higher-order rhythmic structure formed by spatiotemporal integration of meaningful BM information (i.e. the gait cycle of upright walkers rather than inverted walkers) is selectively engaged in the AVI of BM, suggesting that the multisensory processing of natural continuous stimuli may involve unique mechanisms besides the purely stimulus-driven AVI process. These findings advance our understanding of the recently proposed view that multi-timescale neural processes coordinate multisensory integration (Senkowski and Engel, 2024), especially from the perspective of natural stimuli processing. Similar to BM, other natural rhythmic stimuli, like speech and music, also convey hierarchical structures that can entrain neural oscillations at different temporal scales, both in unisensory (Ding et al., 2016; Doelling and Poeppel, 2015) and multisensory contexts (Biau et al., 2022; Crosse et al., 2015; Nozaradan et al., 2016). Possibly, the audiovisual processing of these stimuli engages multi-scale neural coding mechanisms that play distinct functions in perception. Investigating this issue and comparing the results with BM studies will help complete the picture of how the human brain integrates complex, rhythmic information sampled from different sensory modalities to orchestrate perception in natural scenarios.

Since the scope of the current study mainly focused on the neural processing of BM, the task we employed was unrelated to audiovisual correspondence and not for establishing a direct link between neural responses and behavior. This is a limitation of the current study. A recent study demonstrated that listening to frequency-congruent footstep sounds, compared with incongruent sounds, enhanced the visual search for human walkers but not for non-BM stimuli containing the same rhythmic signals, indicating that audiovisual correspondence specifically enhances the perceptual and attentional processing of BM (Shen et al., 2023a). Future research could examine whether the cortical tracking of rhythmic structures plays a functional role in such behaviorally relevant tasks. They could also apply advanced neuromodulation techniques to elucidate the causal relevance of the cortical tracking effect to BM perception (e.g. Kösem et al., 2020; Kösem et al., 2018).

Last but not least, our study demonstrated that the selective cortical tracking of higher-level rhythmic structure in audiovisually congruent BM signals negatively correlated with individual autistic traits. This finding highlights the critical role of the neural tracking of audiovisual BM signals in social cognition. It also offers the first evidence that differences in audiovisual BM processing are already present in nonclinical individuals at the neural level and associated with their autistic traits, extending previous behavioral evidence for atypical audiovisual BM processing in ASD populations (Falck-Ytter et al., 2013; Klin et al., 2009). Meanwhile, given that impaired audiovisual BM processing at the early stage may influence social development and result in cascading consequences for lifetime impairments in social interaction (Falck-Ytter et al., 2018; Klin et al., 2005), it is worth exploring neural tracking of audiovisual BM signals in children, which may pave the way for utilizing it as a potential early neural marker for ASD.

The data and code accompanying this study are made available at https://doi.org/10.57760/sciencedb.psych.00144.

1. 1. Shen L

   (2023) Science Data Bank

   Data from: Cortical tracking of hierarchical rhythms orchestrates the multisensory processing of biological motion.

   https://doi.org/10.57760/sciencedb.psych.00144

1. 1. Arrighi R
   2. Alais D
   3. Burr D

   (2006) Perceptual synchrony of audiovisual streams for natural and artificial motion sequences

   Journal of Vision 6:260–268.

   https://doi.org/10.1167/6.3.6

   • PubMed
   • Google Scholar
2. 1. Baron-Cohen S
   2. Wheelwright S
   3. Skinner R
   4. Martin J
   5. Clubley E

   (2001) The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians

   Journal of Autism and Developmental Disorders 31:5–17.

   https://doi.org/10.1023/a:1005653411471

   • PubMed
   • Google Scholar
3. 1. Bauer AKR
   2. Debener S
   3. Nobre AC

   (2020) Synchronisation of neural oscillations and cross-modal influences

   Trends in Cognitive Sciences 24:481–495.

   https://doi.org/10.1016/j.tics.2020.03.003

   • PubMed
   • Google Scholar
4. 1. Bell AJ
   2. Sejnowski TJ

   (1995) An information-maximization approach to blind separation and blind deconvolution

   Neural Computation 7:1129–1159.

   https://doi.org/10.1162/neco.1995.7.6.1129

   • PubMed
   • Google Scholar
5. 1. Benjamini Y
   2. Hochberg Y

   (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing

   Journal of the Royal Statistical Society Series B 57:289–300.

   https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

   • Google Scholar
6. 1. Biau E
   2. Schultz BG
   3. Gunter TC
   4. Kotz SA

   (2022) Left Motor δ oscillations reflect asynchrony detection in multisensory speech Perception

   The Journal of Neuroscience 42:2313–2326.

   https://doi.org/10.1523/JNEUROSCI.2965-20.2022

   • PubMed
   • Google Scholar
7. 1. Bidet-Caulet A
   2. Voisin J
   3. Bertrand O
   4. Fonlupt P

   (2005) Listening to a walking human activates the temporal biological motion area

   NeuroImage 28:132–139.

   https://doi.org/10.1016/j.neuroimage.2005.06.018

   • PubMed
   • Google Scholar
8. 1. Blake R
   2. Shiffrar M

   (2007) Perception of human motion

   Annual Review of Psychology 58:47–73.

   https://doi.org/10.1146/annurev.psych.57.102904.190152

   • PubMed
   • Google Scholar
9. 1. Brainard DH

   (1997) The psychophysics toolbox

   Spatial Vision 10:433–436.

   https://doi.org/10.1163/156856897X00357

   • PubMed
   • Google Scholar
10. 1. Brooks A
    2. van der Zwan R
    3. Billard A
    4. Petreska B
    5. Clarke S
    6. Blanke O

    (2007) Auditory motion affects visual biological motion processing

    Neuropsychologia 45:523–530.

    https://doi.org/10.1016/j.neuropsychologia.2005.12.012

    • PubMed
    • Google Scholar
11. 1. Covic A
    2. Keitel C
    3. Porcu E
    4. Schröger E
    5. Müller MM

    (2017) Audio-visual synchrony and spatial attention enhance processing of dynamic visual stimulation independently and in parallel: A frequency-tagging study

    NeuroImage 161:32–42.

    https://doi.org/10.1016/j.neuroimage.2017.08.022

    • PubMed
    • Google Scholar
12. 1. Crosse MJ
    2. Butler JS
    3. Lalor EC

    (2015) Congruent visual speech enhances cortical entrainment to continuous auditory speech in noise-free conditions

    The Journal of Neuroscience 35:14195–14204.

    https://doi.org/10.1523/JNEUROSCI.1829-15.2015

    • PubMed
    • Google Scholar
13. 1. Ding N
    2. Melloni L
    3. Zhang H
    4. Tian X
    5. Poeppel D

    (2016) Cortical tracking of hierarchical linguistic structures in connected speech

    Nature Neuroscience 19:158–164.

    https://doi.org/10.1038/nn.4186

    • PubMed
    • Google Scholar
14. 1. Doelling KB
    2. Poeppel D

    (2015) Cortical entrainment to music and its modulation by expertise

    PNAS 112:E6233–E42.

    https://doi.org/10.1073/pnas.1508431112

    • PubMed
    • Google Scholar
15. 1. Duecker K
    2. Doelling KB
    3. Breska A
    4. Coffey EBJ
    5. Sivarao DV
    6. Zoefel B

    (2024) Challenges and approaches in the study of neural entrainment

    The Journal of Neuroscience 44:e1234242024.

    https://doi.org/10.1523/JNEUROSCI.1234-24.2024

    • PubMed
    • Google Scholar
16. 1. Falck-Ytter T
    2. Rehnberg E
    3. Bölte S

    (2013) Lack of visual orienting to biological motion and audiovisual synchrony in 3-year-olds with autism

    PLOS ONE 8:e68816.

    https://doi.org/10.1371/journal.pone.0068816

    • PubMed
    • Google Scholar
17. 1. Falck-Ytter T
    2. Nyström P
    3. Gredebäck G
    4. Gliga T
    5. Bölte S
    6. EASE team

    (2018) Reduced orienting to audiovisual synchrony in infancy predicts autism diagnosis at 3 years of age

    Journal of Child Psychology and Psychiatry, and Allied Disciplines 59:872–880.

    https://doi.org/10.1111/jcpp.12863

    • PubMed
    • Google Scholar
18. 1. Feldman JI
    2. Dunham K
    3. Cassidy M
    4. Wallace MT
    5. Liu Y
    6. Woynaroski TG

    (2018) Audiovisual multisensory integration in individuals with autism spectrum disorder: A systematic review and meta-analysis

    Neuroscience and Biobehavioral Reviews 95:220–234.

    https://doi.org/10.1016/j.neubiorev.2018.09.020

    • PubMed
    • Google Scholar
19. 1. Fleming JT
    2. Noyce AL
    3. Shinn-Cunningham BG

    (2020) Audio-visual spatial alignment improves integration in the presence of a competing audio-visual stimulus

    Neuropsychologia 146:107530.

    https://doi.org/10.1016/j.neuropsychologia.2020.107530

    • PubMed
    • Google Scholar
20. 1. Grossman ED
    2. Blake R

    (2001) Brain activity evoked by inverted and imagined biological motion

    Vision Research 41:1475–1482.

    https://doi.org/10.1016/s0042-6989(00)00317-5

    • PubMed
    • Google Scholar
21. 1. Grossman ED
    2. Battelli L
    3. Pascual-Leone A

    (2005) Repetitive TMS over posterior STS disrupts perception of biological motion

    Vision Research 45:2847–2853.

    https://doi.org/10.1016/j.visres.2005.05.027

    • PubMed
    • Google Scholar
22. 1. Herrmann CS
    2. Murray MM
    3. Ionta S
    4. Hutt A
    5. Lefebvre J

    (2016) Shaping intrinsic neural oscillations with periodic stimulation

    The Journal of Neuroscience 36:5328–5337.

    https://doi.org/10.1523/JNEUROSCI.0236-16.2016

    • PubMed
    • Google Scholar
23. 1. Hosseinian T
    2. Yavari F
    3. Biagi MC
    4. Kuo MF
    5. Ruffini G
    6. Nitsche MA
    7. Jamil A

    (2021) External induction and stabilization of brain oscillations in the human

    Brain Stimulation 14:579–587.

    https://doi.org/10.1016/j.brs.2021.03.011

    • PubMed
    • Google Scholar
24. 1. Jung TP
    2. Makeig S
    3. Westerfield M
    4. Townsend J
    5. Courchesne E
    6. Sejnowski TJ

    (2000) Removal of eye activity artifacts from visual event-related potentials in normal and clinical subjects

    Clinical Neurophysiology 111:1745–1758.

    https://doi.org/10.1016/s1388-2457(00)00386-2

    • PubMed
    • Google Scholar
25. 1. Keitel C
    2. Müller MM

    (2016) Audio-visual synchrony and feature-selective attention co-amplify early visual processing

    Experimental Brain Research 234:1221–1231.

    https://doi.org/10.1007/s00221-015-4392-8

    • PubMed
    • Google Scholar
26. Book

    1. Klin A
    2. Jones W
    3. Schultz RT
    4. Volkmar FR

    (2005)

    The enactive mind-from actions to cognition: lessons from autism

    In: Klin A, Jones W, Schultz RT, Volkmar FR, editors. Handbook of Autism and Pervasive Developmental Disorders: Diagnosis, Development, Neurobiology, and Behavior (3rd ed). John Wiley & Sons Inc. pp. 682–703.

    • Google Scholar
27. 1. Klin A
    2. Lin DJ
    3. Gorrindo P
    4. Ramsay G
    5. Jones W

    (2009) Two-year-olds with autism orient to non-social contingencies rather than biological motion

    Nature 459:257–261.

    https://doi.org/10.1038/nature07868

    • PubMed
    • Google Scholar
28. 1. Kösem A
    2. Bosker HR
    3. Takashima A
    4. Meyer A
    5. Jensen O
    6. Hagoort P

    (2018) Neural entrainment determines the words we hear

    Current Biology 28:2867–2875.

    https://doi.org/10.1016/j.cub.2018.07.023

    • PubMed
    • Google Scholar
29. 1. Kösem A
    2. Bosker HR
    3. Jensen O
    4. Hagoort P
    5. Riecke L

    (2020) Biasing the perception of spoken words with transcranial alternating current stimulation

    Journal of Cognitive Neuroscience 32:1428–1437.

    https://doi.org/10.1162/jocn_a_01579

    • PubMed
    • Google Scholar
30. 1. Laurienti PJ
    2. Perrault TJ
    3. Stanford TR
    4. Wallace MT
    5. Stein BE

    (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies

    Experimental Brain Research 166:289–297.

    https://doi.org/10.1007/s00221-005-2370-2

    • PubMed
    • Google Scholar
31. 1. Ma X
    2. Yuan X
    3. Liu J
    4. Shen L
    5. Yu Y
    6. Zhou W
    7. Liu Z
    8. Jiang Y

    (2022) Gravity-dependent animacy perception in zebrafish

    Research 2022:9829016.

    https://doi.org/10.34133/2022/9829016

    • PubMed
    • Google Scholar
32. 1. Maddox RK
    2. Atilgan H
    3. Bizley JK
    4. Lee AKC

    (2015) Auditory selective attention is enhanced by a task-irrelevant temporally coherent visual stimulus in human listeners

    eLife 4:e04995.

    https://doi.org/10.7554/eLife.04995

    • PubMed
    • Google Scholar
33. 1. Makeig S

    (2002) Response: event-related brain dynamics -- unifying brain electrophysiology

    Trends in Neurosciences 25:390.

    https://doi.org/10.1016/s0166-2236(02)02198-7

    • PubMed
    • Google Scholar
34. 1. Mendonça C
    2. Santos JA
    3. López-Moliner J

    (2011) The benefit of multisensory integration with biological motion signals

    Experimental Brain Research 213:185–192.

    https://doi.org/10.1007/s00221-011-2620-4

    • PubMed
    • Google Scholar
35. 1. Metzger BA
    2. Magnotti JF
    3. Wang Z
    4. Nesbitt E
    5. Karas PJ
    6. Yoshor D
    7. Beauchamp MS

    (2020) Responses to visual speech in human posterior superior temporal gyrus examined with iEEG deconvolution

    The Journal of Neuroscience 40:6938–6948.

    https://doi.org/10.1523/JNEUROSCI.0279-20.2020

    • PubMed
    • Google Scholar
36. 1. Meyer GF
    2. Greenlee M
    3. Wuerger S

    (2011) Interactions between auditory and visual semantic stimulus classes: evidence for common processing networks for speech and body actions

    Journal of Cognitive Neuroscience 23:2291–2308.

    https://doi.org/10.1162/jocn.2010.21593

    • PubMed
    • Google Scholar
37. 1. Nozaradan S
    2. Peretz I
    3. Mouraux A

    (2012a) Selective neuronal entrainment to the beat and meter embedded in a musical rhythm

    The Journal of Neuroscience 32:e122012.

    https://doi.org/10.1523/JNEUROSCI.3203-12.2012

    • PubMed
    • Google Scholar
38. 1. Nozaradan S
    2. Peretz I
    3. Mouraux A

    (2012b) Steady-state evoked potentials as an index of multisensory temporal binding

    NeuroImage 60:21–28.

    https://doi.org/10.1016/j.neuroimage.2011.11.065

    • PubMed
    • Google Scholar
39. 1. Nozaradan S
    2. Schönwiesner M
    3. Caron-Desrochers L
    4. Lehmann A

    (2016) Enhanced brainstem and cortical encoding of sound during synchronized movement

    NeuroImage 142:231–240.

    https://doi.org/10.1016/j.neuroimage.2016.07.015

    • PubMed
    • Google Scholar
40. 1. Obleser J
    2. Kayser C

    (2019) Neural entrainment and attentional selection in the listening brain

    Trends in Cognitive Sciences 23:913–926.

    https://doi.org/10.1016/j.tics.2019.08.004

    • Google Scholar
41. 1. Oostenveld R
    2. Fries P
    3. Maris E
    4. Schoffelen JM

    (2011) FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data

    Computational Intelligence and Neuroscience 2011:156869.

    https://doi.org/10.1155/2011/156869

    • PubMed
    • Google Scholar
42. 1. Pelli DG

    (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies

    Spatial Vision 10:437–442.

    https://doi.org/10.1163/156856897x00366

    • PubMed
    • Google Scholar
43. 1. Saygin AP
    2. Driver J
    3. de Sa VR

    (2008) In the footsteps of biological motion and multisensory perception: judgments of audiovisual temporal relations are enhanced for upright walkers

    Psychological Science 19:469–475.

    https://doi.org/10.1111/j.1467-9280.2008.02111.x

    • PubMed
    • Google Scholar
44. 1. Senkowski D
    2. Engel AK

    (2024) Multi-timescale neural dynamics for multisensory integration

    Nature Reviews. Neuroscience 25:625–642.

    https://doi.org/10.1038/s41583-024-00845-7

    • PubMed
    • Google Scholar
45. 1. Shen L
    2. Lu X
    3. Wang Y
    4. Jiang Y

    (2023a) Audiovisual correspondence facilitates the visual search for biological motion

    Psychonomic Bulletin & Review 30:2272–2281.

    https://doi.org/10.3758/s13423-023-02308-z

    • Google Scholar
46. 1. Shen L
    2. Lu X
    3. Yuan X
    4. Hu R
    5. Wang Y
    6. Jiang Y

    (2023b) Cortical encoding of rhythmic kinematic structures in biological motion

    NeuroImage 268:119893.

    https://doi.org/10.1016/j.neuroimage.2023.119893

    • PubMed
    • Google Scholar
47. 1. Simion F
    2. Regolin L
    3. Bulf H

    (2008) A predisposition for biological motion in the newborn baby

    PNAS 105:809–813.

    https://doi.org/10.1073/pnas.0707021105

    • PubMed
    • Google Scholar
48. 1. Stanford TR
    2. Quessy S
    3. Stein BE

    (2005) Evaluating the operations underlying multisensory integration in the cat superior colliculus

    The Journal of Neuroscience 25:6499–6508.

    https://doi.org/10.1523/JNEUROSCI.5095-04.2005

    • PubMed
    • Google Scholar
49. 1. Stein BE
    2. Stanford TR
    3. Ramachandran R
    4. Perrault TJ
    5. Rowland BA

    (2009) Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness

    Experimental Brain Research 198:113–126.

    https://doi.org/10.1007/s00221-009-1880-8

    • PubMed
    • Google Scholar
50. 1. Stevenson RA
    2. Ghose D
    3. Fister JK
    4. Sarko DK
    5. Altieri NA
    6. Nidiffer AR
    7. Kurela LR
    8. Siemann JK
    9. James TW
    10. Wallace MT

    (2014) Identifying and quantifying multisensory integration: A tutorial review

    Brain Topography 27:707–730.

    https://doi.org/10.1007/s10548-014-0365-7

    • PubMed
    • Google Scholar
51. 1. Thomas JP
    2. Shiffrar M

    (2010) I can see you better if I can hear you coming: action-consistent sounds facilitate the visual detection of human gait

    Journal of Vision 10:14.

    https://doi.org/10.1167/10.12.14

    • PubMed
    • Google Scholar
52. 1. Thomas JP
    2. Shiffrar M

    (2013) Meaningful sounds enhance visual sensitivity to human gait regardless of synchrony

    Journal of Vision 13:13148.

    https://doi.org/10.1167/13.14.8

    • PubMed
    • Google Scholar
53. 1. Troje NF
    2. Westhoff C

    (2006) The inversion effect in biological motion perception: evidence for a “life detector”?

    Current Biology 16:821–824.

    https://doi.org/10.1016/j.cub.2006.03.022

    • PubMed
    • Google Scholar
54. 1. Vallortigara G
    2. Regolin L

    (2006) Gravity bias in the interpretation of biological motion by inexperienced chicks

    Current Biology 16:R279–R80.

    https://doi.org/10.1016/j.cub.2006.03.052

    • PubMed
    • Google Scholar
55. 1. van der Zwan R
    2. Machatch C
    3. Kozlowski D
    4. Troje NF
    5. Blanke O
    6. Brooks A

    (2009) Gender bending: auditory cues affect visual judgements of gender in biological motion displays

    Experimental Brain Research 198:373–382.

    https://doi.org/10.1007/s00221-009-1800-y

    • PubMed
    • Google Scholar
56. 1. Vanrie J
    2. Verfaillie K

    (2004) Perception of biological motion: A stimulus set of human point-light actions

    Behavior Research Methods, Instruments, & Computers 36:625–629.

    https://doi.org/10.3758/BF03206542

    • Google Scholar
57. 1. Wang L
    2. Jiang Y

    (2012) Life motion signals lengthen perceived temporal duration

    PNAS 109:E673–E677.

    https://doi.org/10.1073/pnas.1115515109

    • PubMed
    • Google Scholar
58. 1. Wang L
    2. Yang X
    3. Shi J
    4. Jiang Y

    (2014) The feet have it: local biological motion cues trigger reflexive attentional orienting in the brain

    NeuroImage 84:217–224.

    https://doi.org/10.1016/j.neuroimage.2013.08.041

    • PubMed
    • Google Scholar
59. 1. Wang Y
    2. Zhang X
    3. Wang C
    4. Huang W
    5. Xu Q
    6. Liu D
    7. Zhou W
    8. Chen S
    9. Jiang Y

    (2022) Modulation of biological motion perception in humans by gravity

    Nature Communications 13:e2765.

    https://doi.org/10.1038/s41467-022-30347-y

    • PubMed
    • Google Scholar
60. 1. Wright TM
    2. Pelphrey KA
    3. Allison T
    4. McKeown MJ
    5. McCarthy G

    (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech

    Cerebral Cortex 13:1034–1043.

    https://doi.org/10.1093/cercor/13.10.1034

    • PubMed
    • Google Scholar
61. 1. Wuerger SM
    2. Crocker-Buque A
    3. Meyer GF

    (2012a) Evidence for auditory-visual processing specific to biological motion

    Seeing and Perceiving 25:15–28.

    https://doi.org/10.1163/187847611X620892

    • PubMed
    • Google Scholar
62. 1. Wuerger SM
    2. Parkes L
    3. Lewis PA
    4. Crocker-Buque A
    5. Rutschmann R
    6. Meyer GF

    (2012b) Premotor cortex is sensitive to auditory-visual congruence for biological motion

    Journal of Cognitive Neuroscience 24:575–587.

    https://doi.org/10.1162/jocn_a_00173

    • PubMed
    • Google Scholar

Author details

1. Li Shen

   1. State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
   2. Department of Psychology, University of Chinese Academy of Sciences, Beijing, China

   Contribution

   Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-6088-7892

2. Shuo Li

   1. State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
   2. Department of Psychology, University of Chinese Academy of Sciences, Beijing, China

   Contribution

   Investigation, Writing – original draft

   Competing interests

   No competing interests declared

3. Yuhao Tian

   1. State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
   2. Department of Psychology, University of Chinese Academy of Sciences, Beijing, China

   Contribution

   Investigation, Writing – original draft

   Competing interests

   No competing interests declared

4. Ying Wang

   1. State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
   2. Department of Psychology, University of Chinese Academy of Sciences, Beijing, China

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Writing – review and editing

   For correspondence

   wangying@psych.ac.cn

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5756-2480

5. Yi Jiang

   1. State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
   2. Department of Psychology, University of Chinese Academy of Sciences, Beijing, China

   Contribution

   Conceptualization, Supervision, Writing – review and editing, Funding acquisition

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5746-7301

• 287

  views

• 9

  downloads

• 0

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.