Operating the Grand Capstone Engine

To bridge our manual paper calculations with real-time computational systems, we operate the Tier 3 flagship laboratory: The Complete Vanilla MLP Engine (07_neural_engine_capstone.html).

This Bycroft-grade macro-system synthesizes the entire forward-backward data flow graph at 60 frames per second on a micro-scaled $2 \to 2 \to 1$ architecture, exposing every intermediate tensor register to direct inspection.

Architectural Anatomy of the Capstone Engine

The Capstone Engine models an authentic $2 \to 2 \to 1$ Multi-Layer Perceptron:

[ Input Vector x ] ──► [ Layer 1: W^(1), b^(1) ] ──► [ Hidden a^(1) ] ──► [ Layer 2: W^(2), b^(2) ] ──► [ Prediction y_hat ]
        │                                                                                                  │
        │                                                                                                  ▼
[ Live Update Registers ] ◄── [ Layer 1 Delta δ^(1) ] ◄── [ Layer 2 Delta δ^(2) ] ◄── [ Loss Evaluation L(y, y_hat) ]

Every numerical register in the interface maps directly to a variable in our mathematical derivations:

• Input Registers ($x_1, x_2$): Interactive coordinate sliders defining the input point in 2D space.
• Layer 1 Registers ($W^{(1)} \in \mathbb{R}^{2 \times 2}, b^{(1)} \in \mathbb{R}^{2 \times 1}$): Editable matrix and bias inputs controlling hidden pre-activations $z^{(1)}$ and activations $a^{(1)}$.
• Layer 2 Registers ($W^{(2)} \in \mathbb{R}^{1 \times 2}, b^{(2)} \in \mathbb{R}^{1 \times 1}$): Output weights and hurdle bias computing scalar output $z^{(2)}$ and prediction $\hat{y}$.
• Loss Register ($L$): Real-time quadratic penalty register displaying $L = (\hat{y} - y)^2$.
• Gradient Registers ($\delta^{(2)}, \nabla_{W^{(2)}} L, \delta^{(1)}, \nabla_{W^{(1)}} L$): Exact instantaneous derivatives computed via backpropagation.

Operating Modes & Step-by-Step Verification

The simulator offers two primary modes of operation:

Mode 1: Single-Step Step-by-Step Execution

1. Click Step Forward (1) to execute Layer 1 affine transformations ($z^{(1)} = W^{(1)}x + b^{(1)}$). Watch the intermediate registers update.
2. Click Step Forward (2) to apply activation functions ($a^{(1)}$) and evaluate Layer 2 ($z^{(2)}, \hat{y}$).
3. Click Evaluate Loss to compute mistake penalty $L = (\hat{y} - y)^2$ against target $y$.
4. Click Step Backward (Output) to compute output delta $\delta^{(2)}$ and output gradients $\frac{\partial L}{\partial W^{(2)}}, \frac{\partial L}{\partial b^{(2)}}$.
5. Click Step Backward (Hidden) to backpropagate error through $(W^{(2)})^T \delta^{(2)}$ and compute hidden gradients $\frac{\partial L}{\partial W^{(1)}}, \frac{\partial L}{\partial b^{(1)}}$.
6. Click Apply Parameter Update with step size $\eta$ to execute $W_{\text{new}} = W_{\text{old}} - \eta \nabla L$. Inspect the updated registers and verify that loss immediately decreases on the next forward evaluation.

Mode 2: Continuous 60fps Flow

Enable continuous mode to observe real-time tensor streaming. Dragging input sliders or parameter knobs causes every downstream register and gradient arrow to re-evaluate deterministically at sub-millisecond speeds.

Curated Presets and Hypothesis Exploration

To test critical boundary conditions, load the curated presets from the simulator toolbar:

1. Preset 1: Bivariate Non-Linear Separation

   • Configuration: A 2D XOR-like classification problem with two positive diagonal points and two negative diagonal points.
   • Inquiry: Observe how Hidden Neuron 1 and Hidden Neuron 2 partition the 2D coordinate space into two linear half-spaces, folding the plane so the output layer achieves linear separation.

2. Preset 2: Flop Pitch Error Correction

   • Configuration: Initial weights set to $0.50$, input $x = [1.0, 2.0]^T$, target $y = 0.0$, step size $\eta = 0.50$.
   • Inquiry: Verify that the simulator's intermediate registers match your hand-calculated paper derivations: $\hat{y} = 0.6937$, $L_1 = 0.4812$, $\delta^{(2)} = 0.2948$, $\delta^{(1)} = [0.0220, 0.0220]^T$, and updated loss $L_2 = 0.3774$.

3. Preset 3: Symmetric Initialization Saddle Point

   • Configuration: All Layer 1 weights initialized to identical values ($W_{11} = W_{12} = W_{21} = W_{22} = 0.50$) with identical inputs.
   • Inquiry: Observe that hidden error deltas $\delta_1^{(1)}$ and $\delta_2^{(1)}$ receive identical gradients. This demonstrates why symmetric weight initialization causes hidden neurons to learn identical features, proving why distinct initial weights are necessary.

Guided Challenge: "Execute a single manual forward and backward cycle on paper and verify every intermediate tensor and gradient against the Capstone Engine's live registers."

Strict Scope Containment Boundary

We deliberately stop our software and mathematical modeling at this single manual step.

To maintain clarity and protect cognitive bandwidth, the following complex engineering mechanisms are reserved for subsequent specialized courses:

• No Complex Optimizers: We use only the foundational step rule $\Delta W = -\eta \nabla L$ on paper; no advanced momentum-based or adaptive rate optimizers.
• No Automated Multi-Iteration Training Loops: No automated iteration loops, convergence checks, or dataset iterators.
• No Dataset Chunking Schemes: We evaluate single isolated feature vectors on paper without mini-grouping or dataset partitioning.
• No Hardware Accelerators or Framework Libraries: No hardware acceleration engines, specialized parallel processing kernels, or automatic differentiation software libraries.

Mastering the complete forward-backward data flow graph by hand guarantees that when you encounter automated deep learning libraries later, every internal tensor transformation is already transparent.