MuzzammilShah/NeuralNetworks-LanguageModels-5

Deep MLP with WaveNet-inspired Architecture: Makemore (Part 5)

This repository builds upon the 2-layer MLP from previous implementations, making it deeper with a tree-like structure, leading to a convolutional neural network architecture inspired by the WaveNet (2016) from DeepMind.

Overview

• This hierarchical architecture is implemented efficiently using causal dilated convolutions (not yet covered).
• The project explores torch.nn and its functionality, providing insights into deep learning development processes.

Documentation

For a better reading experience and detailed notes, visit my Road to GPT Documentation Site.

Acknowledgments

Notes and implementations inspired by the Makemore - Part 5 video by Andrej Karpathy.

For more of my projects, visit my Portfolio Site.