Topic 4 The Concept Of Time And Sequence Order Introduction
Master the formal proof of permutation equivariance in self-attention. Compare O(n) recurrence with O(1) path lengths and analyze the need for positional encoding.
Content adapted from Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin.Original Source