Dot Products and Weights

The dot product is the single most fundamental mathematical operation in neural networks. It is how models evaluate inputs, measure influence, and calculate decisions.

Calculating the Grocery Bill

You have your grocery list (vector $x$). Now you go to the store. The store has a list of prices for Apples, Bananas, and Carrots. We can represent the prices as another vector, $w$:

$$w = \begin{bmatrix} \char36 4 \\ \char36 1 \\ \char36 2 \end{bmatrix} \begin{matrix} \text{(Price per Apple)} \\ \text{(Price per Banana)} \\ \text{(Price per Carrot)} \end{matrix}$$

To find your total bill, you perform a position-wise multiplication.

Because both vectors share the exact same locked positions (Apples at Pos 1, Bananas at Pos 2, Carrots at Pos 3), you simply multiply Position 1 by Position 1, Position 2 by Position 2, and so on:

• Pos 1 (Apples): $2 \times \char36 4 = \char36 8$
• Pos 2 (Bananas): $3 \times \char36 1 = \char36 3$
• Pos 3 (Carrots): $1 \times \char36 2 = \char36 2$

Summing these individual products produces a single scalar number:

$$\text{Total Bill} = \char36 8 + \char36 3 + \char36 2 = \mathbf{\char36 13}$$

You have just computed a Dot Product, written mathematically as $w \cdot x$:

$$w \cdot x = (4 \times 2) + (1 \times 3) + (2 \times 1) = 13$$

Formal Definition ($n$-Dimensions)

The grocery bill isn't a special case — it's exactly what the dot product means for any two vectors of matching dimension $n$:

$$w \cdot x = \sum_{i=1}^{n} w_i x_i = w_1 x_1 + w_2 x_2 + \cdots + w_n x_n$$

In words: multiply the two vectors position-by-position, then sum every product into a single scalar.

Because the dot product multiplies elements pairwise, both vectors must have the exact same dimension. If vector $x$ has 3 elements and vector $w$ has 4, the 4th element has no partner to multiply with, making the operation mathematically undefined.

Weights and Influence

Notice that in our grocery math, we used the letter $w$ for the prices. In neural networks, $w$ stands for Weights.

The word "weight" is not random mathematical jargon; it literally refers to how much weight or influence a feature carries:

• A positive weight: If a feature has a positive corresponding weight (like $+10.0$), it influences the final score to be higher, favoring a positive decision.
• A negative weight: If a feature has a negative corresponding weight (like $-2.0$), it influences the final score to be lower, favoring a negative decision.
• A zero weight: And if a feature has a corresponding weight of $0.0$, it exerts no influence on the final score—it carries zero weight, even if the feature value itself is a huge number.

An individual weight in the weight vector is simply the exact numerical measure of how much influence the corresponding feature has on the final decision.

This is why the dimensions of the feature vector ($x$) and the weight vector ($w$) must always match in AI models: every feature needs an influence value to determine its impact on the final score—even if that influence is zero ($0.0$).

Applied Scenario: The Netflix Greenlight Predictor

Let's see this in action using a 4-dimensional vector. Netflix is building an algorithm to evaluate incoming scripts for Hit Potential, comparing two scripts: 'Die Hard in Space' and 'The Notebook 2'.

First, the algorithm loads the Audience Influence Weights ($w$) across the locked genre order [Action, Romance, Comedy, Sci-Fi]. This represents what Netflix believes audiences currently care about, which directly influences whether a movie has strong hit potential:

$$w = \begin{bmatrix} 5.0 \\ -2.0 \\ 1.0 \\ 4.0 \end{bmatrix} \begin{matrix} \text{(Action Weight)} \\ \text{(Romance Weight)} \\ \text{(Comedy Weight)} \\ \text{(Sci-Fi Weight)} \end{matrix}$$

Audiences currently love Action ($+5.0$) and Sci-Fi ($+4.0$), mildly enjoy Comedy ($+1.0$), and strongly dislike Romance ($-2.0$).

Now, the algorithm evaluates the Movie Genre Vectors ($x$) of the two scripts:

$$x_{\text{Die Hard}} = \begin{bmatrix} 1.0 \\ 0.0 \\ 0.5 \\ 1.0 \end{bmatrix} \qquad\qquad x_{\text{Notebook 2}} = \begin{bmatrix} 0.0 \\ 1.0 \\ 0.5 \\ 0.0 \end{bmatrix}$$

To visualize how the dot product engine evaluates a pitch, observe how the inputs and weights flow through parallel multiplication channels into a single decision node:

Each horizontal path computes a single term of our formal equation ($w_i \cdot x_i$), measuring the exact boost or penalty contributed by that genre. The central summation node ($\sum$) collects all four streams and collapses them into a single scalar value ($+9.5$).

Below is the complete arithmetic breakdown for both scripts:

🎬 'Die Hard in Space': Raw Score

Feature	Feature Value (x)	Weight (w)	Product (wi · xi)	Dot Product (w · x)
Action	1.0	+5.0	+5.0	+9.5
Romance	0.0	−2.0	0.0
Comedy	0.5	+1.0	+0.5
Sci-Fi	1.0	+4.0	+4.0

🎬 'The Notebook 2': Raw Score

Feature	Feature Value (x)	Weight (w)	Product (wi · xi)	Dot Product (w · x)
Action	0.0	+5.0	0.0	−1.5
Romance	1.0	−2.0	−2.0
Comedy	0.5	+1.0	+0.5
Sci-Fi	0.0	+4.0	0.0

'Die Hard in Space' produces a strongly positive score ($+9.5$), whereas 'The Notebook 2' receives a negative score ($-1.5$). Relative to each other, the dot product is already telling us something real: current audience tastes favor the action pitch far more heavily than romance.

🔬 Laboratory: The Dot Product and Weights

Test mathematical boundary conditions to observe how weights control output values across positive, negative, and zero inputs.

Key Mathematical Takeaways from the Laboratory:

• Zero-Weight Nullification: Setting any weight $w_i = 0$ completely zeroes out that feature's term ($0 \times x_i = 0$), proving that a feature carries zero influence regardless of how large its value is.
• The Balance of Power: The final score is not determined by large numbers alone, but by the net competition between positive boosts ($\sum w_i x_i > 0$) and negative penalties ($\sum w_i x_i < 0$).
• Unbounded Scale: Notice that as you dial multiple weights to maximum values, the score can grow arbitrarily large (e.g., $+15.0$ or higher), demonstrating why raw dot products cannot serve as final probabilities without an activation function.

---