Multi-Neuron Layer Architecture

Across the last three lessons, we assembled the complete machinery to run multiple artificial neurons in parallel:

1. The Weight Matrix (Lesson 1): We stacked neuron weights into rows of a matrix ($W$) to compute parallel raw scores ($Wx$).
2. The Bias Vector (Lesson 2): We bundled independent baseline offsets into a vector ($b$) to produce linear scores ($z = Wx + b$).
3. Vector Activation (Lesson 3): We applied the Sigmoid function element-wise to transform linear scores into independent probabilities ($a = \sigma(z)$).

When you group multiple independent neurons together to evaluate the exact same input vector in parallel, you form the fundamental building block of deep learning: a Neural Network Layer.

What Is a "Layer"?

A layer is a collection of independent neurons operating in parallel on the exact same input vector ($x$).

• Modeling complexity: A single neuron can only evaluate one perspective. Using multiple neurons allows the system to analyze the exact same input from multiple angles simultaneously, letting us model far more complex real-world problems.
• Each neuron is isolated: The Hit neuron has no idea the Oscar neuron exists. Each neuron has its own weights (a row in $W$), its own baseline bias (an entry in $b$), and produces its own activation. You could calculate each neuron on a separate piece of paper without changing the math.
• One output per neuron: The number of neurons in a layer always matches the number of entries in its output vector. Because we have 3 neurons, our layer outputs a 3-dimensional vector ($a$), with each entry corresponding to its neuron.
• Layer Width: The number of neurons in a layer defines the layer's width. Because our layer has 3 neurons evaluating the script, its width is 3.
• Parallel execution: If each neuron is independent, why use a matrix? A matrix lets hardware compute every neuron simultaneously, rather than running them one by one.

A layer doesn't change how each neuron works—the matrix just packages them together so they run in parallel.

The Unified Forward Pass

When we assemble the weights, biases, and activations into a single computational pipeline, we get the complete forward pass for a multi-neuron layer:

In a single compact formula:

Tracing the End-to-End Pipeline

🎬 'Die Hard in Space': Full Pipeline Trace

🎬 'The Notebook 2': Full Pipeline Trace

The Missing Piece: The Greenlight Dilemma

Our multi-neuron layer evaluated both scripts and produced three independent answers:

• 'Die Hard in Space': Commercial Hit: Yes ($99.9\%$), Oscar Contender: No ($0.02\%$), International Appeal: Yes ($>99.9\%$).
• 'The Notebook 2': Commercial Hit: No ($2.9\%$), Oscar Contender: Yes ($62.2\%$), International Appeal: Yes ($50.0\%$).

But answering these questions still doesn't decide whether to greenlight a film. The studio is left weighing trade-offs between conflicting answers: does massive overseas appeal make up for zero awards buzz? Is an Oscar contender worth funding if it loses money domestically?

Because neurons in a single layer operate in total isolation, they cannot combine their answers. To automate the final verdict—Greenlight or Pass—we need another layer that takes these three outputs as its input before making the final decision.

Parallel neurons gave our network width—evaluating multiple criteria at once. But automating decisions across those criteria requires depth—stacking layers sequentially into a Multi-Layer Neural Network. That is the focus of Topic 4.