Module & Parameter
Module trees, gradients, and modes.
01Common interface#
forward(const Tensor&) and backward(const Tensor&) are pure virtual. New modules start in training mode. train/eval propagate to registered children. eval is not a no-grad mode.
02Learnable values#
Parameter holds data, grad, and requires_grad. grad starts at zero. requires_grad=false stops that parameter derivative; input gradients still compute. accumulate_grad_locked merges replicas outside the step and checks element count only.
03Registration and traversal#
parameters returns raw pointers; named_parameters adds hierarchical names. register_parameter records weights, register_buffer nonlearnable state, register_module children. Repeated registration of the same parameter is not deduplicated.
04Your own layer#
Accumulate parameter derivatives with +=. Keep working buffers in the object and respect the latest-forward cache lifetime.
class Scale : public Module { public: Tensor forward(const Tensor& x) override { y_.resize_like(x); for (std::size_t i=0;i<x.numel();++i) y_[i]=2*x[i]; return y_; } Tensor backward(const Tensor& dy) override { dx_.resize_like(dy); for (std::size_t i=0;i<dy.numel();++i) dx_[i]=2*dy[i]; return dx_; } private: Tensor y_, dx_;};