RNN Foundations: Recurrence, State, and O(n) Bottlenecks
Master the math of RNNs, LSTMs, and GRUs. Understand hidden state updates, the O(n) sequential bottleneck, hardware constraints, and the limitations of BPTT.
Content adapted from Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin.Original Source