Manual Backpropagation through BatchNorm: Makemore (Part 4)
This repository explores manual backpropagation through a 2-layer MLP (with BatchNorm) without using PyTorch autograd's loss.backward(). It involves manually backpropagating through the cross entropy loss, 2nd linear layer, tanh, batchnorm, 1st linear layer, and the embedding table.
Documentation
For a better reading experience and detailed notes, visit my Road to GPT Documentation Site.
Acknowledgments
Notes and implementations inspired by the Makemore - Part 4 video by Andrej Karpathy.
For more of my projects, visit my Portfolio Site.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support