Construction of flight control models for singly presented visual patterns.

(A) Schematic of the experimental setup (left), a frame captured by the infrared camera (middle), and a simplified schematic of the experimental setup (right). The annulus surrounding the fly schematic represents the visual display viewed from above. (B) Wing responses of a sample fly to three rotating visual patterns: bar, spot, and grating. L-R WBA at the bottom row represents the angular torque of the fly, calculated by subtracting the right wingbeat amplitude (RWBA) from the left wingbeat amplitude (LWBA). (C) L-R WBA traces of a sample fly in response to the three visual patterns. The thick black lines indicate the average of all trials (top) or the average of all flies (bottom). Thin gray lines indicate individual trials (top) or fly averages (bottom). (D) Schematic of the position-velocity-based flight control model. (E) Average wing responses of a population of flies in response to the three rotating visual patterns, either in a clockwise (red) or counterclockwise (blue) direction. Top traces show the position of each pattern. The red and blue shadings at the bottom indicate the 95% confidence interval. (F) Position and velocity functions estimated from wing responses in E. Light purple shading indicates the 95% confidence interval.

The flight control models with a bio-mechanics block predicted the dynamics of the orientation behavior to individual visual patterns.

(A) Schematics of three visuomotor response models with a biomechanics block. (B) Simplified version of the position and velocity responses for each pattern. (C) Simulation results for the three patterns (bar, spot, and grating) moving in a sigmoidal dynamics. The spot response was plotted with an 180° offset to facilitate comparison. Bar plots on the right show the latency of body angle with respect to the stimulus onset, measured at the 50% point of the pattern movement. (D) Simulation results for the three patterns moving in a sinusoidal dynamics. In the bar plots on the right, the amplitude was measured as the peak-to-peak amplitude, and the phase shift was calculated by measuring the peak time of the cross-correlation between the pattern and the fly heading. (E) Same as in C, but for visual patterns remaining static at 0 degree position. The simulation was performed 10 times with a gaussian noise component (gray lines). The mean response was plotted in thick colored lines. The probability density function of the body angle is shown on the right for each pattern.

Magnetically tethered flight experiments confirmed orientation changes predicted by the virtual fly model.

(A) Schematic of the magnetically tethered flight assay with an LED display (left). The image acquired from below was analyzed to estimate the body angle (right). The stimulus protocol (bottom) consisted of four phases: alignment, ready, go, and freeze. (B) Body orientation responses of a single fly for the bar, spot and grating patterns moving horizontally. (C) Same as in B for a population of flies. The population averages were replotted at the bottom to facilitate the comparison of their dynamics. (D) Amplitude and latency of the body orientation responses. The box represents the interquartile range (IQR), with the median indicated by the horizontal black line. The whiskers extend to the minimum and maximum values within 1.5 times the IQR. Outliers are denoted by “+” marks beyond the whiskers.

Three integrative models of the visuomotor control and their predictions in a complex visual environment.

(A) Schematic of the visual environments used in the simulation. A moving bar is presented as a foreground object over the static grating background. (B) A diagram of the addition-only model. Bar and grating response circuits are joined at their output by addition. (C) A diagram of the graded EC model. An EC block translates the bar-evoked motor command to the negative image of the predicted grating input to counteract visual feedback. (D) A diagram of the all-or-none EC model. An EC switches off the grating response circuit during the bar-evoked turn. (E,F,G) Simulation results of the three models. The bar position and the heading of the virtual fly model (top). The associated torques as well as the EC signal (bottom).

Behavior of EC-based models in changing visual backgrounds.

(A) Two different visual environments, one with few objects in the background representing a weak visual feedback and the other with many objects such as trees representing a strong visual feedback. (B) A diagram of the auto-tuned, graded EC model, where a multi-layered perceptron (MLP) is used to adapt the EC amplitude to the changing visual feedback. The loss function is computed based on the mean squared error (MSE) between the angular position of the fly when the efference copy amplitude matches that of the visual feedback. (C) Simulation results for a moving bar with a static background over iterations when the visual feedback increases at the 400th iteration. The top plot depicts the change of the feedback amplitude and the EC amplitude, the second panel the MSE, and the third panel the 50% latency of the heading response. The bottom plot show the sample traces before and after the feedback change. (D) Same as in C, but when the feedback decreases at the 400th iteration. (E) Same as in C, but for the all-or-none EC model.

Dynamics of object-evoked flight turns were not affected by the background pattern.

(A) Temporal profile of the visual stimulus used to test the bar response when the background changes from the uniform to the dense-dot background. In each frame, the horizontal midline of the display is sampled and plotted over time (top). Sample pattern images at the moment of the bar movement onset (bottom middle). (B) Body angle changes to the rotation of the random dot backgrounds in two different densities (left). The amplitude is significantly larger for the dense background than the sparse one (right, Wilcoxon rank sum test). (C) Average body orientation traces in response to horizontally moving bars when the initial uniform, bright background is kept the same or changed to one of the two random-dot backgrounds. The thick colored lines in the top plots represent the population average, whereas the gray lines represent an average for individual flies. Box-and-whisker plots depict the amplitude (middle) and latency (bottom) of the body angle change. Error bars (bottom) indicate 95% confidence interval. (D) Same as in (C), but for different combinations of background patterns. (E) Schematic of the stimulus protocol for testing adaptation in the bar response with a maintained background. The background was maintained in either sparse or dense dot patterns for 7 minutes (“priming phase”) and then changed to a different density. After this change, the same background pattern was maintained for 14 minutes (“test phase” #1 and #2). (F) Population-averaged body angle traces in response to a moving bar in three different phases.

(Related to Fig. 1) An expanded model predicts the history-dependent dynamics in the wing responses.

(A) Schematic of an expanded flight control model with a motor-wing dynamics and a fatigue block. (B) A heat map indicating the errors in the grating responses for different combinations of the fatigue block parameters. (C) Visually evoked wing responses to the three rotating patterns and the predictions of the two models. (D) Position and velocity responses of the simple and expanded models.

(Related to Fig. 1) Construction of flight control models for Canton S flies.

(A) Schematic of the experimental setup (left), a frame captured by the infrared camera (middle), and a simplified schematic of the experimental setup (right). The annulus surrounding the fly schematic represents the visual display viewed from above. (B) L-R WBA traces of a sample fly in response to the bar and spot visual patterns. The thick black lines indicate the average of all trials (top) or the average of all flies (bottom). Thin gray lines indicate individual trials (top) or fly averages (bottom). (C) Schematic of the position-velocity-based flight control model. (D) Position and velocity functions estimated from wing responses in C. Light purple shading indicates 95% confidence interval.

(Related to Fig. 2) The response latency of the flight control model.

(A) Same as in Fig. 2A with an additional delay block. (B) Same as in Fig. 2C comparing delayed and non-delayed heading of the virtual fly model for each visual pattern. (C) Same as in Fig. 2C for different peak amplitudes of the sigmoid-like pattern trajectory and different frequencies of the sine-like trajectory. Plots on the right show the change of delay with respect to the stimulus amplitude (top) and the frequency of the sine stimulus (bottom two).

(Related to Fig. 4) Orientation behaviors of the two efference copy-based models for different stimulus conditions.

(A) Diagrams of the graded and all-or-none EC models. (B) The change in the EC signals with respect to the bar evoked flight torque. (C) Heading and torque responses to four different stimulus conditions: a moving bar in the first step, a moving bar and grating leftwards in the second step, a moving bar rightwards and grating leftwards in the third step, and a moving grating in the fourth step. (D) Bar plots indicating the amplitude of the response and the latency with respect to the stimulus onset, measured at the 50% point of the pattern movement for the different stimulus combinations. Dotted lines were added to facilitate the comparison across conditions. The all-or-none EC model exhibited less variability than the graded EC model.

(Related to Fig. 6) Unexpected changes of the background pattern did not affect loom-evoked behavioral responses.

(A) Temporal change of the visual stimulus profile along the horizontal midline of the display. Body angle responses are shown in Fig. 6D. (B) Head angle responses to the stimulus profile in A with four different combinations of background patterns. The head angle was calculated from DeepLabCut. (C) On the left, schematic of the experimental display showing the initial position of the loom stimulus. On the right, stimulus profile and background combinations. (D) Body angle responses to the stimulus profile in C, for the different background combinations. (E) Same as in D for different background combinations.