A layered model of fly walking that incorporates link-and-joint dynamics and sensorimotor delays.

(A) The neural systems that control walking are hierarchically organized. The brain sends high-level commands (e.g. walking speed and direction) to the ventral nerve cord (VNC). Approximately 70 motor neurons (MNs, Azevedo et al. (2022)) control each leg. The cell bodies and dendrites of the MNs are in the VNC, and their axons innervate muscles in the leg. Each leg muscle is actuated, after some delay, following activation by one or more motor neurons. Sensorimotor circuits in the VNC also receive delayed sensory feedback via proprioceptors. (B) Multi-layered walking model architecture. Body-world interactions are mediated by a dynamics model. Proprioceptive feedback consists of joint angles and angular velocities (Mamiya et al., 2018). Each leg has an optimal controller operating at 600 Hz that interfaces with the dynamics model and a trajectory generator that generates realistic kinematics. The trajectory generator is learned from data and operates at 300 Hz. It interfaces with the phase coordinator, a Kuramoto oscillator that maintains inter-leg coupling. The trajectory generator and optimal controller mimic local circuits within each leg, so they do not interact with other legs. Phase coupling is the only information shared between all legs. (C) Dynamics model of a fly leg, derived from link-and-joint models and Euler-Lagrange equations. (D) Schematic of model training and execution process. Model parameters for each module are tuned independently, then the modules are assembled to generate walking with delays and in response to unexpected perturbations.

Walking model simulations produce realistic joint kinematics.

(A) Example time-series of femur-tibia flexion (R1) and femur rotation (L2) for three different walking speeds: 8, 10, and 12 mm/s. Real data (orange) exhibited more variability than simulations (blue). (B) Angle vs. computed per-leg phase of femur-tibia flexion on leg R1 and femur rotation on leg L2 for four different walking speeds. Each plot contains data from 4 walking bouts with different initial conditions. (C) Average differences between model simulations and data, over a range of forward walking, turning, and side-stepping speeds over 500 distinct bouts. The dotted line (5.56 degrees) indicates uncertainty associated with markerless 3D joint tracking (Karashchuk et al., 2021). All simulations used a sensory delay of 10 ms and motor delay of 30 ms, based on values measured experimentally with electrophysiology from leg sensory and motor neurons/muscles (Tuthill and Wilson, 2016a; Azevedo et al., 2020). neurons/muscles (Tuthill and Wilson, 2016a; Azevedo et al., 2020).

The model sustains robust walking during and following perturbations.

(A) Schematic of how perturbations are applied to the walking model. (B, C) Example time-series of femur-tibia flexion on leg R1 and femur rotation on leg L2 for three different walking speeds (8, 10, and 12 mm/s) before, during, and after perturbation. In panel B, an impulse perturbation of size 3.75 rad/s is applied at a single instant (dotted line), and its effects are analyzed over a brief time window (shaded blue area). In panel C, stochastic perturbations of size 1.875 rad/s are applied over a time window (shaded green area). Angle trajectories are visibly different during the analysis windows, but recover afterward. Perturbation effects appear similar across speeds. All simulations used a sensory delay of 10 ms and motor delay of 30 ms.

The walking model sustains realistic walking kinematics following impulse and persistent stochastic perturbations.

(A) Method for computing kinematic similarity between real data and simulated walking. First, the full set of experimental data (from flies) is used to compute a Gaussian kernel density estimator (KDE). To quantify the similarity (to data) of a given bout of simulated walking, we apply the KDE to evaluate the log probability density function of each bout, a scalar value we refer to as kinematic similarity (KS). High KS indicates that the perturbed walking resembles the unperturbed walking from data, while low KS indicates that the perturbed walking deviates from data. The average KS value of bouts from experimental data was -1.627. (B, C) Kinematic similarity of simulated walking during perturbations for impulse and persistent stochastic perturbations. For each square of the heatmap, four simulations with different initial conditions were simulated and averaged. Simulations became less similar to data with increased perturbation strength. All simulations used a sensory delay of 10 ms and motor delay of 30 ms. Perturbation velocities were drawn from a random uniform distribution with mean perturbation strength (rad/s) and standard deviation of 0.1×mean.

The model maintains robust walking under persistent stochastic perturbations over specific ranges of motor and sensory delays.

(A, B) Example time-series of femur-tibia flexion on leg R1 and femur rotation on leg L2 under various values of motor (10, 20, 30, 40 ms) and sensory delay (0, 5, 10, 15 ms). Perturbation effects became more noticeable with increasing delay values. (C, D, E) Similarity of during-perturbation walking to data across delay values, perturbation strengths, and forward speeds. For each square of the heatmap, four simulations with different initial conditions were simulated and evaluated. At low delay values, simulated walking maintained high similarity even under large perturbations. As perturbation strength and delays increased, simulated walking became less similar to data; the effect was more pronounced with increased delays. When we fixed one delay value and vary the other, the model maintained realistic walking (KS > −1.6) up to about 30 ms of motor delay and 10 ms of sensory delay across a range of conditions. When we allowed both motor and sensory delay to vary, the model maintained realistic walking when the sum of the delays is no more than about 45 ms. (F) Post-perturbation walking with motor and sensory delays. The model was unable to recover from perturbations for large delay values. Unless otherwise stated, forward speed = 12 mm/s, and perturbation strength = 1.875 rad/s.

Joints included for leg models

Velocity and phase distributions in data justify selection of model simulation parameters.

(A) Cumulative distribution functions and number of frames associated with various forward, rotation, and side-step velocities in data. Rotation (i.e. yaw) and right/left walking peaked at 0 mm/s, while forward/backward walking achieved a sustained peak from approximately 4 mm/s to 10 mm/s. Most model-generated simulations in the paper adhered to this range. (B) Rotation vs. side-step velocity. Each dot on the scatter plot represents a single bout from data. Rotation velocity and side-step velocity were highly correlated with one another in data. (C) Phase offsets between legs in data over a range of walking velocities and rotational velocities. Side-step velocities were omitted from study since (as previously demonstrated) they were highly correlated with rotation velocity. Phase offsets remained relatively constant over a range of forward and rotational velocities.

Simulated walking reproduced realistic joint angles.

Angle vs. computed per-leg phase of all joints on all legs for four different walking speeds. All simulations used a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values. Some oddities were observed in real data, for instance in L3 femur flexion and R1 body-coxa flexion. The model does not quite capture these oddities.

Simulated walking reproduced realistic joint angular velocities.

Angular velocity vs. computed per-leg phase of all joints on all legs for four different walking speeds. All simulations used a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values.

Simulated walking resembled data over a range of forward, turning, and side-stepping velocities.

Average differences between model simulations over 500 distinct bouts. All simulations use a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values. The dotted line (5.56 degrees) indicates uncertainty associated with data collection.

Simulated walking reproduced across- and within-leg coupling characteristics of data.

(A) Phase coupling across legs. We compared phases of representative joints across legs. Model coupling qualitatively resembled data coupling, with moderate variations in phase difference (i.e. peak location) and synchronization strength (i.e. peak height). A strong peak in probability density indicates synchronization, whereas broader peaks indicate weak synchronization. A single peak at zero on the horizontal axis indicates that the two joint phases are coupled to match; a single peak elsewhere indicates that the two joint phases are coupled with some phase offset. Simulations were performed over a range of forward walking, turning, and side-stepping speeds over 500 distinct bouts. (B, C) Phase coupling within each leg. For each leg, we compared phases between a representative joint for the leg (denoted “target” on the image) and other joints on the leg. Gaps are present as we did not include all 5 joints for all legs in the model. All synchronizations (i.e. peaks) in data are reproduced by the model. However, simulations generally exhibited stronger synchronization (i.e. higher peaks).

Model generated robust walking under persistent stochastic perturbations over select ranges of motor and sensory delays, revealing fundamental constraints on delay values.

(A, B) Example simulated time-series of femur-tibia flexion on leg R1 and femur rotation on leg L2 under various values of motor (10, 20, 30, 40 ms) and sensory delay (0, 5, 10, 15 ms). Perturbation effects were more noticeable with increased delay values. (C, D, E) KS of simulated during-perturbation walking to data, for various values of delay, perturbation strength, and forward speeds. For each square of the heatmap, four simulations with different initial conditions were simulated and evaluated. As perturbation strength and delays increased, simulated walking became less similar to data; the effect was more pronounced with increased delays. When we fixed one delay value and varied the other, The model maintained realistic walking (KS > −1.6) up to about 25 ms of motor delay and 8 ms of sensory delay across a range of conditions. When we allowed both motor and sensory delay to vary, the model maintained realistic walking when the sum of the delays was no more than about 40 ms. (F) Post-perturbation walking with motor and sensory delays. The model is able to recover from perturbations for very large values of delay. Unless otherwise stated, forward speed = 12 mm/s, and perturbation strength = 5 rad/s.