CUDA & MatrixFlash
Device pointers, fused kernels, and GEMM bridge.
01Host / device boundary#
NEXUS_MODEL_WITH_CUDA compiles device functions. Host Tensor is not transferred automatically. The caller owns cudaMalloc, transfers, synchronization, error checks, and cudaFree. This API does not provide device-side backward.
02Fused interfaces#
Softmax fuses max/exp/sum/division; LayerNorm fuses mean/variance/normalize/scale/shift. All pointers must refer to device memory, with positive rows/cols.
namespace nexus_model::cuda { void relu(const float* x,float* y,int n); void gelu_tanh(const float* x,float* y,int n); void fused_softmax(const float* x,float* y,int rows,int cols); void fused_layernorm(const float* x,const float* gamma, const float* beta,float* y,int rows,int cols,float eps);}03MatrixFlash bridge#
With WITH_MATRIXFLASH, bridge/matrixflash.hpp provides gemm(left,right) around matrix_pro::multiply. CPU GEMM belongs to NexusModel. This header does not automatically change Linear forward; consumers call it explicitly.
04Measurement scope#
GPU numbers are cudaEvent timings with device-resident data. They exclude transfers, allocation, backward, and the full training pipeline. GEMM benchmark calls cuBLAS directly; it is not a NexusModel GPU GEMM implementation.