LLM Foundations: Architecture & Inference

Deconstruct modern large language models from first principles, covering subword tokenization, self-attention mechanisms, and transformer inference.

What is Covered

• Tokenization Mechanics: Byte-Pair Encoding (BPE) and embedding vectors.
• Attention Engines: Query-Key-Value matrices, scaled dot-product attention, and multi-head projection.
• Inference Pipeline: Autoregressive decoding, KV caching, and sampling strategies.