ltc_dynamics (macula_tweann v0.13.0)
View SourceLiquid Time-Constant (LTC) neural dynamics.
This module implements Liquid Time-Constant neurons and their Closed-form Continuous-time (CfC) approximation for high-performance evaluation.
LTC neurons have input-dependent time constants, enabling adaptive temporal processing. This makes them particularly suitable for time-series data and real-time control tasks.
Theory
The LTC neuron is governed by the ODE:
dx(t)/dt = -[1/tau + f(x,I,theta)] * x(t) + f(x,I,theta) * A
Where: - x(t) is the neuron's internal state at time t - tau is the base time constant (learnable) - f(...) is a nonlinear function producing the "liquid" time constant - A is the state bound (prevents explosion) - I(t) is the input at time t
CfC Approximation
Instead of solving the ODE numerically, CfC uses a closed-form approximation:
x(t+dt) = sigma(-f) * x(t) + (1 - sigma(-f)) * h
Where: - sigma is the sigmoid function - f is the backbone network output (time-constant modulator) - h is the head network output (target state)
CfC is approximately 100x faster than ODE-based LTC while maintaining similar expressivity.
References
[1] Hasani, R., Lechner, M., et al. (2021). "Liquid Time-constant Networks." Proceedings of the AAAI Conference on Artificial Intelligence.
[2] Hasani, R., Lechner, M., et al. (2022). "Closed-form Continuous-time Neural Networks." Nature Machine Intelligence.
[3] Beer, R.D. (1995). "On the Dynamics of Small Continuous-Time Recurrent Neural Networks." Adaptive Behavior, 3(4).
Summary
Functions
Clamp state to bounds [-Bound, Bound].
Compute backbone network output (time-constant modulator).
Compute head network output (target state).
Compute the liquid (input-dependent) time constant.
Evaluate CfC neuron with closed-form approximation.
Evaluate CfC neuron with custom backbone/head weights.
Evaluate LTC neuron using ODE integration (Euler method).
Evaluate LTC with ODE and custom parameters.
Reset state to zero.
Sigmoid function.
Hyperbolic tangent function.
Types
Functions
Clamp state to bounds [-Bound, Bound].
Bounded dynamics are a key property of LTC that ensures stability.
-spec compute_backbone(input(), tau(), backbone_weights()) -> float().
Compute backbone network output (time-constant modulator).
The backbone determines how the time constant varies with input. This is the "f" in the CfC equation.
-spec compute_head(input(), head_weights()) -> float().
Compute head network output (target state).
The head determines the asymptotic state the neuron moves toward. This is the "h" in the CfC equation.
Compute the liquid (input-dependent) time constant.
The "liquid" aspect of LTC comes from the time constant varying based on input and state, allowing adaptive temporal dynamics.
Evaluate CfC neuron with closed-form approximation.
This is the fast path for LTC evaluation, avoiding ODE solving. Uses default backbone/head functions based on input only.
Evaluate CfC neuron with custom backbone/head weights.
For networks with evolved backbone/head parameters.
Evaluate LTC neuron using ODE integration (Euler method).
More accurate than CfC but slower. Use for training or when temporal precision is critical.
Evaluate LTC with ODE and custom parameters.
-spec reset_state() -> state().
Reset state to zero.
Sigmoid function.
sigma(x) = 1 / (1 + e^-x)
Hyperbolic tangent function.