Explicit backward

The analytic derivative and layer-local tape decision.

2 min readv1.0.0C++20

From the implementationinclude/nexus_model/core/details.txt

01Path B#

MatrixFlash Variable::backward seeds the root with ones and accepts no external dL/dy. Sibling libraries do not share a tape. NexusModel uses an explicit-gradient backward interface.

02Differentiate without storing a Jacobian#

Linear, Conv, Pool, Norm, Dropout, Embedding, activations, and RNN/LSTM/GRU implement analytic vector-Jacobian products. Sequential traverses in reverse. The attention softmax/matmul chain uses MicroTape.

\frac{\partial L}{\partial x_i}=\sum_j\frac{\partial y_j}{\partial x_i}\frac{\partial L}{\partial y_j}

03Caching and accumulation#

Parameters receive contributions from multiple paths, so derivatives add. Input derivatives belong to that backward call. Reusing a layer object for another forward in a graph overwrites its cache. Use separate modules or replicas.

◆Read alongside the source. API behavior reflects the current implementation.

02Differentiate without storing a Jacobian#

\frac{\partial L}{\partial x_i}=\sum_j\frac{\partial y_j}{\partial x_i}\frac{\partial L}{\partial y_j}