Figures and data in A neuronal least-action principle for real-time learning in cortical circuits

Figures
Tables
Additional files

6 figures, 1 table and 1 additional file

Figures

Figure 1

Download asset Open asset

Somato-dendritic mismatch energies and the neuronal least-action (NLA) principle.

(a1) Sketch of a cross-cortical network of pyramidal neurons described by NLA. (a2) Correspondence between elements of NLA and biological observables such as membrane voltages and synaptic weights. (b1) The NLA principle postulates that small variations $δ \tilde{𝒖}$ (dashed) of the trajectories $\tilde{𝒖}$ (solid) leave the action invariant, $δ A = 0$ . It is formulated in the look-ahead coordinates $\tilde{𝒖}$ (symbolized by the spyglass) in which `hills' of the Lagrangian (shaded gray zones) are foreseen by the prospective voltage so that the trajectory can turn by early enough to surround them. (b2) In the absence of output nudging ( $β = 0$ ), the trajectory $𝒖 (t)$ is solely driven by the sensory input, and prediction errors and energies vanish ( $L = 0$ , outer blue trajectory at bottom). When nudging the output neurons towards a target voltage ( $β > 0$ ), somatodendritic prediction errors appear, the energy increases (red dashed arrows symbolising the growing ‘volcano’) and the trajectory $𝒖 (t)$ moves out of the $L = 0$ hyperplanes, riding on top of the `volcano' (red trajectory). Synaptic plasticity $\dot{W}$ reduces the somatodendritic mismatch along the trajectory by optimally ‘shoveling down the volcano’ (blue dashed arrows) while the trajectory settles in a new place on the $L = 0$ hyperplane (inner blue trajectory at bottom).

Figure 2

Download asset Open asset

Prospective coding in cortical pyramidal neurons enables instantaneous voltage-to-voltage transfer.

(a1) The instantaneous spike rate of cortical pyramidal neurons (top) in response to sinusoidally modulated noisy input current (bottom) is phase-advanced with respect to the input adapted from Köndgen et al., 2008. (a2) Similiarly, in neuronal least-action (NLA), the instantaneous firing rate of a model neuron ( $r = ρ (u) + τ \dot{ρ} (u)$ , black) is phase-advanced with respect to the underlying voltage ( $u$ , red, postulating that the low-pass filtered rate is a function of the voltage, $\overline{r} = ρ (u)$ ). (b) Dendritic input in the apical tree (here called $\overline{e}$ ) is instantaneously causing a somatic voltage modulation ( $u$ , modeling data from Ulrich, 2002). The low-pass filtering with $τ$ along the dendritic shaft is compensated by a lookahead mechanism in the dendrite ( $e = \overline{e} + τ \dot{\overline{e}}$ ). In (Ulrich, 2002) a phase advance is observed even with respect to the dendritic input current, not only the dendritic voltage, although only for slow modulations (as here). (c) While the voltage of the first neuron ( $u_{1}$ ) integrates the input rates $r_{in}$ from the past (bottom black upward arrows), the output rate $r_{1}$ of that first neuron looks ahead in time, $r_{1} = ρ (u_{1}) + τ \dot{ρ} (u_{1})$ (red dashed arrows pointing into the future). The voltage of the second neuron ( $u_{2}$ ) integrates the prospective rates $r_{1}$ (top black upwards arrows). By doing so, it inverts the lookahead operation, resulting in an instantaneous transfer from $u_{1} (t)$ to $u_{2} (t)$ (blue arrow and circles).

Figure 3

Download asset Open asset

Moving equilibrium hypothesis for motor control and real-time learning of cortical activity.

(a) A voluntary movement trajectory can be specified by the target length of the muscles in time, ${𝒖_{𝒐}}^{*}$ , encoded through the $γ$ -innervation of muscle spindles, and the deviation of the effective muscle lengths from the target, $𝒖_{𝒐} - {𝒖_{𝒐}}^{*} = - {\overline{𝒆}}_{𝒐}^{*}$ . The $I_{a}$ -afferents emerging from the spindles prospectively encode the error, so that their low-pass filtering is roughly proportional to the length deviation, truncated at zero (red). The moving equilibrium hypothesis states that the low-pass filtered input ${\overline{𝒓}}_{in}$ , composed of the movement plan ${\overline{𝒓}}_{in}^{plan}$ and the sensory input (here encoding the state of the plant e.g., through visual and proprioceptive input, ${\overline{𝒓}}_{in}^{vis}$ and ${\overline{𝒓}}_{in}^{prop}$ ), together with the low-pass filtered error feedback from the spindles, ${\overline{𝒆}}_{𝒐}^{*}$ , instantaneously generate the muscle lengths, $𝒖_{𝒐} = 𝑭_{W} ({\overline{𝒓}}_{in}, {\overline{𝒆}}_{𝒐}^{*})$ , and are thus at any point in time in an instantaneous equilibrium (defined by Equation 7a, Equation 7b). (b1) Intracortical intracortical electroencephalogram (iEEG) activity recorded from 56 deep electrodes and projected to the brain surface. Red nodes symbolize the 56 iEEG recording sites modeled alternately as input or output neurons, and blue nodes symbolize the 40 ‘hidden’ neurons for which no data is available, but used to reproduce the iEEG activity. (b2) Corresponding NLA network. During training, the voltages of the output neurons were nudged by the iEEG targets (black input arrows, but for all red output neurons). During testing, nudging was removed for 14 out of these 56 neurons (here, represented by neurons 1, 2, 3). (c1) Voltage traces for the 3 example neurons in a2, before (blue) and after (red) training, overlaid with their iEEG target traces (gray). (c2) Total cost, integrated over a window of 8 s of the 56 output nodes during training with sequences of the same duration. The cost for the test sequences was evaluated on a 8 s window not used during training.

Figure 4

Download asset Open asset

On-the-fly learning of finger responses to visual input with real-time dendritic error propagation (rt-DeEP).

(a) Functionally feedforward network with handwritten digits as visual input ( $𝒓_{in}^{(2)} (t)$ in Figure 3a, here from the MNIST data set, 5 ms presentation time per image), backprojections enabling credit assignment, and activity of the 10 output neurons interpreted as commands for the 10 fingers (forward architecture: 784×500×10 neurons). (b) Example voltage trace (b1) and local error (b2) of a hidden neuron in neuronal least-action (NLA) (red) compared to an equivalent network without lookahead rates (orange). Note that neither network achieves a steady state due to the extremely short input presentation times. Errors are calculated via exact backpropagation, i.e., by using the error backpropagation algorithm on a pure feedforward NLA network at every simulation time step (with output errors scaled by $β$ ), shown for comparison (blue dots). (c) Comparison of network models during and after learning. Color scheme as in (b). (c1) The test error under NLA evolves during learning on par with classical error backpropagation performed each Euler $d t$ based on the feedforward activities. In contrast, networks without lookahead rates are incapable of learning such rapidly changing stimuli. (c2) With increasing presentation time, the performance under NLA further improves, while networks without lookahead rates stagnate at high error rates. This is caused by transient, but long-lasting misrepresentation of errors following stimulus switches: when plasticity is turned off during transients and is only active in the steady state, comparably good performance can be achieved (dashed orange). (d) Receptive fields of 6 hidden-layer neurons after training, demonstrating that even for very brief image presentation times (5ms), the combined neuronal and synaptic dynamics are capable of learning useful feature extractors such as edge filters.

Figure 5

Download asset Open asset

Hierarchical plastic microcircuits implement real-time dendritic error learning (rt-DeEL).

(a) Microcircuit with ‘top-down’ input (originating from peripheral motor activity, blue line) that is explained away by the lateral input via interneurons (dark red), with the remaining activity representing the error ${\overline{e}}_{l}$ . Plastic connections are denoted with a small red arrow and nudging with a dashed line. (b1) Simulated network with 784-300-10 pyramidal-neurons and a population of 40 interneurons in the hidden layer used for the MNIST learning task where the handwritten digits have to be associated with the 10 fingers. (b2) Test errors for rt-DeEL with joint tabula rasa learning of the forward and lateral weights of the microcircuit. A similar performance is reached as with classical error backpropagation. For comparability, we also show the performance of a shallow network (dashed line). (b3) Angle derived from the Frobenius norm between the lateral pathway ${𝑾^{IP}}_{l} {𝑾^{PI}}_{l}$ and the feedback pathway $𝑩_{l} 𝑾_{l + 1}$ . During training, both pathways align to allow correct credit assignment throughout the network. Indices are dropped in the axis label for readability.

Appendix 1—figure 1

Download asset Open asset

Recovering presynaptic potentials through short term depression.

(a1) Relative voltage response of a depressing cortical synapse recreated from Abbott et al., 1997, identified as synaptic release probability $p$ . (a2) The product of the low-pass filtered presynaptic firing rate $\bar{r} (u)$ times the synaptic release probability is $p (\bar{r})$ proportional to the presynaptic membrane potential, $p (\bar{r}) \bar{r} \propto u$ . (a3) Average in vivo firing rate of a neuron in the visual cortex as a function of the somatic membrane potential recreated from Anderson et al., 2000, which can be qualitatively identified as the stationary rate $\bar{r} (u)$ derived in Equation 43.

Tables

Table 1

Mathematical symbols.

Mathematical expression	Naming	Comment
$u_{i}$	Instantaneous (somatic) voltage	only for network neurons
$r_{i} = ρ (u_{i}) + τ \dot{ρ} (u_{i})$	Instantaneous firing rate of neuron i	that looks linearly ahead in time
$\bar{r} (t) = \frac{1}{τ} \int_{- \infty}^{t} r (t^{'}) e^{- \frac{t - t^{'}}{τ}} d t^{'}$	Definition of low-pass filtering	See Equation 15
${\bar{r}}_{i} = ρ (u_{i}) = \bar{r_{i} + τ {\dot{r}}_{i}}$	Low-pass filtered firing rate	postulated to be a function of $u_{i}$
$𝒓 = \bar{𝒓} + τ \dot{\bar{𝒓}}$	Self-consistency eq.	for low-pass filtered rate
$𝒓_{in}$	Input rate vector, column	projects to selected neurons
${\bar{𝒓}}_{in}$	Low-pass filter input rates	instantaneously propagates
$e_{i} = (u_{i} + τ {\dot{u}}_{i}) - \sum_{j} W_{i j} r_{j}$	Prospective error of neuroni	in apical dendrite
${\bar{e}}_{i} = u_{i} - \sum_{j} W_{i j} {\bar{r}}_{j}$	Error of neuroni	in soma
${E^{M}}_{i} = \frac{1}{2} {\bar{e}}_{i}^{2} = \frac{1}{2} (u_{i} - \sum_{j} W_{i j} {\bar{r}}_{j})^{2}$	Mismatch energy in neuron i	between soma and basal dendrite
$u_{o}^{*}$	Target voltage for output neuron o	could impose target on $r_{o}$ or ${\bar{r}}_{o}$
${\bar{e}}_{o}^{} = u_{o}^{} - u_{o}$	Error of output neuron o	also called target error
$C_{o} = \frac{1}{2} {({\bar{e}}_{o}^{*})}^{2}$	Cost contribution of output neuron o	between soma and basal dendrite
$L = \sum_{i \in N} {E^{M}}_{i} + β \sum_{o \in O} C_{o}$	Lagrangian	$output O \subset network N$
$\tilde{u} (t) = \frac{1}{τ} \int_{t}^{\infty} u (t^{'}) 𝒆^{(t - t^{'}) / τ} d t^{'}$	Discounted future voltage	prospective coordinates for NLA
$𝒖 = \tilde{𝒖} - τ \dot{\tilde{𝒖}}$	Self-consistency eq.	for discounted future voltage
$A = \int_{t_{1}}^{t_{2}} L [\tilde{𝒖} (t), \dot{\tilde{𝒖}} (t)] d t$	Neuronal Least Action (NLA)	expressed in prospect. coordinates
$\frac{\partial L}{\partial {\tilde{u}}_{i}} - \frac{d}{d t} \frac{\partial L}{\partial {\dot{\tilde{u}}}_{i}} = (1 + \frac{d}{d t}) \frac{\partial}{\partial u_{i}} L = 0$	Euler-Lagrange equations	turned into lookahead operator
$𝑾_{in}$	weights from input neurons $𝒓_{in}$	$dim (N) \times dim (𝒓_{in})$ , most0
$𝑾_{net}$	weights between network neurons	$dim (N) \times dim (N)$
$𝑾 = (𝑾_{in}, 𝑾_{net})$	total weight matrix	$dim (N) \times (dim (𝒓_{in}) + dim (N))$
$𝒓 = {(𝒓_{in}, 𝒓_{net})}^{T}$	instantaneous firing rate vector	column (indicated by transpose)
$\dot{𝑾} \propto \bar{𝒆} {\bar{𝒓}}^{T}$	Plasticity of $𝑾$	$\bar{e}$ is a column, ${\bar{𝒓}}^{T}$ a row vector
$𝒖_{𝒐}^{} (t) = 𝑭^{} ({\bar{𝒓}}_{in} (t))$	Target function formulated for ${\bar{r}}_{in} (t)$	a functional of $𝒓_{in} (t)$
$𝒖_{𝒐} (t) = 𝑭_{W} ({\bar{𝒓}}_{in} (t), {\bar{𝒆}}_{𝒐}^{*} (t))$	Func. implemented by forward network	instant. func. of ${\bar{𝒓}}_{in} (t)$ , not $𝒓_{in} (t)$
$N$	Layers in forward network, w/o $r_{in}$	Last-layer voltages: $𝒖_{N} = 𝒖_{𝒐}$
$W_{l}^{I P}$	Weights from pyr to interneurons	lateral, within layer l
$W_{l}^{P I}$	Weights from inter- to pyr’neurons	lateral, within layer l
$𝑾_{l}$	Bottom-up weights from layerl–1 tol	between pyramidal neurons
$𝑩_{l}$	Top-down weights from layerl+1 tol	between pyramidal neurons
${\bar{e}}_{l}^{A} = B_{l} u_{l + 1} - {W^{PI}}_{l} {u^{I}}_{l}$	Low-pass filtered apical error in layerl	top-down minus lateral feedback
${\bar{e}}_{l} = {\bar{r}}_{l}^{'} \cdot {\bar{e}}_{l}^{A} = {\bar{r}}_{l}^{'} \cdot B_{l} {\bar{e}}_{l + 1}$	Somato-basal prediction error	is correct error for learning
$E_{l}^{I P} = \frac{1}{2} ‖ u_{l}^{I} - W_{l}^{IP} {\bar{r}}_{l} ‖^{2}$	Interneuron mismatch energy	minimized to learn $W_{l}^{I P}$
$E_{l}^{P I} = \frac{1}{2} ‖ B_{l} u_{l + 1} - {W^{PI}}_{l} {u^{I}}_{l} ‖^{2}$	Apical mismatch energy	minimized to learn $W_{l}^{P I}$
$η, η^{IP}, η^{PI}$	Learning rates for plasticity of…	… $W_{l}; W_{l}^{I P}; W_{l}^{P I}$
$𝑯 = \frac{\partial^{2} L}{\partial 𝒖^{2}} = 𝟏 - 𝑾_{net} 𝝆^{'} - {\bar{𝒆}}^{'}$	Hessian, $\frac{\partial^{2} L}{\partial 𝒖^{2}} = \frac{\partial 𝒇}{\partial 𝒖}$ . If pos. definite	⇒ stable dynamics
$𝒇 (𝒖, t) = \frac{\partial L}{\partial 𝒖} = 𝒖 - 𝑾 \bar{𝒓} (𝒖) - \bar{𝒆} (𝒖)$	Corrected error	becomes 0 with $τ$
$𝒇 (𝒖, t) + τ \dot{𝒇} (𝒖, t) = 0$	Euler-Lagrange equations	satisfy $f (u, t) = f_{0} e^{- (t - t_{0}) / τ}$
$𝒇 (𝒖, t) = 0$	Always the case after transient	exponentially decaying with $τ$
$\dot{𝒖} = - \frac{1}{τ} 𝑯^{- 1} (𝒖) (𝒇 (𝒖) + τ \frac{\partial 𝒇}{\partial t})$	Explicit diff. eq.	obtained by solving for $\dot{𝒖}$
$𝒈 (𝒖, t) = - \frac{1}{τ} 𝑯^{- 1} (𝒖) (𝒇 (𝒖) + τ \frac{\partial 𝒇}{\partial t})$	Used to write the explicit diff. eq.	$\dot{𝒖} = 𝒈 (𝒖, t)$
$𝑮 (𝒚, \dot{𝒖}) = (1 + τ \frac{d}{d t}) \frac{\partial L}{\partial 𝒖} = 𝒇 + τ \dot{𝒇}$	Used for contraction anaylsis, Equation 53	$𝒚 = (𝒓_{in}, 𝒖_{𝒐}^{*}, 𝒖)$
$𝑴, 𝑲$	Used to iteratively converge to $\dot{𝒖}$	see Equation 46
$\overset{˘}{𝒖} = 𝒖 + τ \dot{𝒖}$	Linear lookahead voltage	Latent Equilibrium, Appendix 4

Additional files

MDAR checklist: https://cdn.elifesciences.org/articles/89674/elife-89674-mdarchecklist1-v1.pdf
Download elife-89674-mdarchecklist1-v1.pdf

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Article PDF

Open citations (links to open the citations from this article in various online reference manager services)

Mendeley

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Walter Senn
Dominik Dold
Akos F Kungl
Benjamin Ellenberger
Jakob Jordan
Yoshua Bengio
João Sacramento
Mihai A Petrovici

(2024)

A neuronal least-action principle for real-time learning in cortical circuits

eLife 12:RP89674.

https://doi.org/10.7554/eLife.89674.3

Share this article

Cite this article

Somato-dendritic mismatch energies and the neuronal least-action (NLA) principle.

Prospective coding in cortical pyramidal neurons enables instantaneous voltage-to-voltage transfer.

Moving equilibrium hypothesis for motor control and real-time learning of cortical activity.

On-the-fly learning of finger responses to visual input with real-time dendritic error propagation (rt-DeEP).

Hierarchical plastic microcircuits implement real-time dendritic error learning (rt-DeEL).

Recovering presynaptic potentials through short term depression.

Mathematical symbols.

MDAR checklist

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)