What a neuron actually computes

Strip away the biology metaphors and the diagrams with glowing circles. An artificial neuron is one multiplication, one addition, and one squash. That is the entire thing. Everything else in deep learning is that operation, repeated.

The one equation

A neuron takes some numbers in and produces one number out. Here is the whole computation, for a neuron with two inputs:

Four ingredients, and every one of them has a plain meaning:

• x₁, x₂ — the inputs. Whatever you are feeding the neuron. Pixel brightness, a person's age, the previous layer's output. Just numbers.
• w₁, w₂ — the weights. How much the neuron cares about each input. A weight of 3 means "this input matters three times as much". A weight of −2 means "when this input goes up, my answer should go down". A weight of 0 means "ignore this entirely".
• b — the bias. The neuron's default leaning before it sees anything. A large positive bias means an eager neuron that fires unless talked out of it; a large negative bias means a sceptical one.
• f — the activation function. A squashing step applied at the very end. Without it, stacking neurons would be pointless — we'll prove that in a moment.

The part inside the brackets, w₁x₁ + w₂x₂ + b, is called the weighted sum or pre-activation, usually written z. It is a dot product plus a constant, nothing more.

A worked example you should do by hand

Suppose we are building a neuron that decides whether to recommend a film. Two inputs:

• x₁ = the film's rating out of 10, divided by 10 → a number between 0 and 1
• x₂ = 1 if it is longer than three hours, 0 otherwise

Say the neuron has learned w₁ = 4.0 (rating matters a lot), w₂ = −1.5 (long films are a mild negative), and b = −2.0 (start out sceptical). Now a film rated 8.5/10 that runs three and a half hours:

Pass that through a sigmoid activation, f(z) = 1 / (1 + e^−z), and you get 0.475. The neuron says: 47.5% — a marginal no. Shorten the film to under three hours (x₂ = 0) and z becomes 1.4, giving 0.80. A confident yes.

Lab: draw the boundary yourself

Here is the geometric truth that most explanations skip. Because z is a linear function of the inputs, the set of points where z = 0 — where the neuron is exactly undecided — is a straight line. Everything on one side gets classified as "yes", everything on the other as "no".

Move the sliders below. The line is the neuron's decision boundary. Try to separate the orange dots from the blue ones.

Three things the lab should have shown you

1. The weights control the angle of the line. The vector (w₁, w₂) points perpendicular to the boundary, in the direction of "more yes". Double both weights and the line does not move at all — only the sharpness of the transition changes.

2. The bias slides the line without rotating it. That is its only job. Without a bias term, the boundary would be forced through the origin, and a huge number of problems would become unsolvable for no good reason.

3. XOR cannot be done. Select the XOR dataset and try every combination you like. You will not exceed 75% accuracy. The four points are arranged so that no single straight line separates them — and a single neuron only ever gives you one straight line.

Why the activation function is not optional

Here is a proof you can follow in three lines. Suppose we drop f and stack two layers:

Substitute the first into the second:

Let W* = W₂W₁ and b* = W₂b₁ + b₂. You are left with y = W*x + b* — a single linear layer. A hundred stacked linear layers collapse into one. The depth buys you literally nothing.

Insert any non-linear f between the layers and that collapse becomes impossible. That is the entire reason activation functions exist. They are not a biological flourish; they are what makes depth mean something.

The code

Twenty lines, no framework. Open the playground and run it.

# A single neuron, from scratch
import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def neuron(x, w, b):
    z = np.dot(w, x) + b       # weighted sum
    return sigmoid(z)          # squash to (0, 1)

# the film-recommender from earlier
w = np.array([4.0, -1.5])
b = -2.0

for rating, is_long in [(0.85, 1), (0.85, 0), (0.40, 0)]:
    x = np.array([rating, is_long])
    p = neuron(x, w, b)
    print(f"rating={rating}, long={is_long} -> {p:.3f}")

# rating=0.85, long=1 -> 0.475
# rating=0.85, long=0 -> 0.802
# rating=0.4,  long=0 -> 0.401

Check yourself

1. A neuron has w = [2, −1] and b = 0.5. What is z for the input x = [1, 3]? What does the sigmoid of that come to, roughly? (z = −0.5; σ ≈ 0.38)
2. If you multiply every weight and the bias by 10, does the decision boundary move? Does anything change at all? (The line stays put; predictions become far more confident — the transition from 0 to 1 gets much sharper.)
3. Why can a neuron with a bias of 0 never classify the point (0, 0) as "yes" with confidence above 0.5? (Because z = 0 there, and σ(0) = 0.5 exactly.)
4. Sketch, on paper, a two-neuron hidden layer that solves XOR. Hint: one neuron learns OR, the other learns NAND, and the output neuron learns AND.

Where this goes next

You now understand a single unit. The next lesson stacks them into layers and shows what the forward pass looks like as matrix multiplication. After that comes backpropagation — the algorithm that finds good weights automatically instead of you dragging sliders.