Topic 2 The Computational Bottleneck Of Traditiona Introduction
Analyze the Softmax Bottleneck in NLMs. Learn to scale vocabularies using Hierarchical Softmax, reducing complexity from O(V) to O(log V) via binary trees.
Content adapted from Efficient Estimation of Word Representations in Vector Space by Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean.Original Source