What a neural network actually is

If you're a software engineer, you've spent your career writing programs the same way: decompose a problem into rules, encode those rules in a language, ship a deterministic artifact that executes them. The function is_valid_email(s) does what it does because you (or someone) wrote down what "valid" means.

A neural network is a different kind of artifact. It's still code that runs on a computer. It still has inputs and outputs. But nobody wrote down the rules. The "rules" — to the extent the word applies — live as several billion floating-point numbers that were fit from examples by an optimizer. The architecture is the part a human designs. The behavior is the part the optimizer fills in.

Andrej Karpathy called this Software 2.0, and I think it's the most honest framing for engineers. "Software 2.0 is written in much more abstract, human unfriendly language, such as the weights of a neural network," he wrote in November 2017. "No human is involved in writing this code because there are a lot of weights (typical networks might have millions), and coding directly in weights is kind of hard (I tried)." The dataset and architecture are the source code; training is the compiler; the trained network is the binary.

That's the executive summary. The rest of this post unpacks it. We'll build from one artificial "neuron" up through layers, the forward pass, the loss, and gradient descent — enough that the rest of this series (on training, inference, hallucinations, knowledge cutoffs) has somewhere to land. Light math, no derivations.

The smallest unit

The basic building block is laughably simple. It takes a vector of input numbers, multiplies each by a weight, adds them up, adds a bias, and passes the result through a nonlinear function. In math:

output = f(w₁·x₁ + w₂·x₂ + … + wₙ·xₙ + b)

The sliders below wire that up directly. Try dragging the bias deep into the negative — even with strong positive inputs, ReLU floors the output at zero. Switch the activation to "None" to strip away the nonlinearity and see the raw weighted sum. Switch it back to see the kink.

What the bias controls is worth pausing on: it's the neuron's resting threshold — the gravitational pull toward silence. A weight of zero erases an input entirely; a deeply negative bias means the remaining inputs have to collectively overcome something before the neuron fires at all. That's what it means for a neuron to "specialize" during training: it learns not just what to amplify, but how much evidence it needs before it speaks.

To see this in domain terms, here's a single-neuron email spam classifier. Three features go in: spam keyword count, exclamation marks, and whether the sender is in your address book. Each gets a weight; the sum gets a bias. The only change from the widget above is the activation — sigmoid instead of ReLU, because here we want the output as a probability between 0 and 1, not a clipped activation signal. Same structure, different semantics.

A few things worth sitting with. The known-sender weight is −3.0: one trust signal strong enough to pull the score below the spam threshold even if the email has multiple keywords. That's a learned belief that provenance outweighs vocabulary. The bias of −2.0 means an email with no features at all scores sigmoid(−2.0) ≈ 0.12 — a gentle "probably fine" prior that evidence has to actively overcome in either direction.

And this is still one neuron. Real filters stack thousands, each tuned to fire on a different combination, their outputs passed forward and recombined until a verdict emerges from the last layer. The one above is not a toy simplification — it's the literal unit, running the same math. The question of why sigmoid here but ReLU everywhere else is worth holding onto; the next paragraph answers it.

The weights wᵢ and the bias b are the things we eventually learn. f is a fixed function called the activation. In modern networks it's almost always ReLU — Rectified Linear Unit — which is just max(0, x). Negative numbers become zero; positive numbers pass through. That's it. (Vinod Nair and Geoffrey Hinton popularized ReLU in 2010.)

Why ReLU instead of the older sigmoid? Because sigmoid neurons saturate for large inputs, producing near-zero gradients that multiply to nearly nothing across deep layers — the vanishing gradient problem. ReLU has a constant gradient of 1 for all positive inputs, so the error signal propagates undiminished through a deep stack. It's cheaper to compute and trains much better.

People call this unit a "neuron" because the original 1940s and 50s researchers — McCulloch, Pitts, Rosenblatt — were inspired by a cartoon of how brain cells work. Don't take the metaphor too seriously. A real biological neuron is a wet electrochemical system with thousands of dendrites, spike timing, and neuromodulators we still don't fully understand. The artificial "neuron" is a dot product followed by max(0, x). It is to a biological neuron roughly what a paper airplane is to a 787. Treat the name as historical residue, not a claim about biology.

Why the nonlinearity at all? Because without it, the whole network — no matter how many layers — collapses into a single linear function. Stacking linear operations gives you another linear operation. The kink in ReLU is what lets the network bend space, and bending space is what lets it represent things like "this pixel pattern is a 7."

If you want the cleanest one-line definition of an individual neuron, borrow Grant Sanderson's (3Blue1Brown): a neuron is "a thing that holds a number." The whole network is a bunch of those, wired together so that the numbers in one layer determine the numbers in the next.

Stacking them: layers and the forward pass

A layer is a bunch of these units side by side, all looking at the same inputs but with different weights. You can describe an entire layer with one operation: multiply the input vector by a weight matrix, add a bias vector, apply the nonlinearity elementwise.

output = f(Wx + b)

That Wx + b is the part where matrix multiplication earns its central place in deep learning. If your input is a length-784 vector (say, a flattened 28×28 image of a handwritten digit) and your first layer has 128 units, then W is a 128×784 matrix and the layer does one matrix-vector multiply, one vector add, and one elementwise ReLU. That's the whole computation, repeated for the next layer, and the next, until the output.

A forward pass is exactly this chain of matrix multiplies and nonlinearities running once, end to end, for one input. For a classifier with 10 output classes (digits 0–9), the final layer might have 10 units, and the output is interpreted as "how strongly does this network think the input is each digit?"

This means a neural network, at runtime, is a very specific kind of computation graph: a sequence of dense linear algebra operations with cheap nonlinearities between them. That's why GPUs are so good at it. A GPU is a machine that multiplies large matrices fast; a neural network is something that needs large matrices multiplied fast.

The flip side: a network with a few billion parameters is mostly just a few billion floating-point numbers sitting in memory, sliced into matrices, waiting to be multiplied by an input. There is no hidden cleverness in how it executes. The cleverness is entirely in which numbers got chosen.

The widget below makes that concrete. It runs a 4 → 5 → 4 → 2 network on three email examples — a classic spam message, a legitimate newsletter, and a borderline promotional email. Use the stepper to walk the signal through one layer at a time. The network has been trained already; you're watching it compute.

The borderline case is the most instructive. No hidden layer forms a decisive pattern — a few neurons fire weakly, without conviction — and that ambiguity carries all the way to the output. A real classifier signals "I don't know" not by refusing to answer, but by arriving at nearly equal scores.

The weights are the program

This is the conceptual leap. In a traditional codebase, the program's behavior lives in source code: branches, loops, function definitions. In a neural network, the program's behavior lives in the weights. The architecture (how many layers, how wide, what nonlinearity, how they connect) is a skeleton. The weights are the muscle.

You can take the same architecture, train it on a dataset of cat-vs-not-cat images, and you get a cat detector. Train the same architecture on a dataset of English-to-French sentence pairs, and you get a translator. Same skeleton; different weights; entirely different program. The dataset is, in a real sense, the source code. Training is the compiler. The trained network is the binary.

For an engineer, two consequences are worth absorbing:

1. You don't read this code. Inspecting individual weights tells you nothing, the way reading a single bit of a compiled binary tells you nothing. Interpretability researchers like Chris Olah have built impressive tools to make trained networks legible, but in everyday practice the network is opaque-by-construction.
2. Bugs live in the data and the architecture. When a neural network misbehaves, you usually don't fix it by editing weights. You fix it by changing the dataset, the loss, or the architecture, and recompiling — i.e., retraining.

Most of your engineering instincts about deterministic state, code review, and version control apply to the training pipeline, not to the model itself.

How does it learn?

Start with random weights. Now we need a way to tell the network it's wrong, and a way to nudge the weights toward being less wrong.

The first piece is a loss function. It takes the network's output and the correct answer and produces a single number — how badly the network screwed up on this example. For a classifier, the standard loss is cross-entropy; for regression, mean squared error. The details don't matter for this post. What matters is that the loss is differentiable: a tiny change in any weight produces a tiny, measurable change in the loss.

That property is everything. Because the loss is differentiable, we can compute its gradient with respect to every weight — a vector of partial derivatives that says, for each knob in the system, "if you nudge me this way, the loss goes up; if you nudge me that way, it goes down." Then we take a small step in the down direction for every weight at once. That step is called gradient descent, and the chain-rule trick that lets us compute the gradient efficiently through a deep network is called backpropagation (Rumelhart, Hinton & Williams, 1986).

I won't derive backprop here — that's its own post. The shape of the loop is what matters:

1. Sample a batch of training examples.
2. Run the forward pass to get the network's outputs.
3. Compute the loss against the true answers.
4. Compute the gradient of the loss with respect to every weight (backprop).
5. Nudge every weight a small amount in the direction that reduces the loss.
6. Repeat. Millions of times. Billions, for big models.

The widget below compresses that loop to two weights. Pick a landscape — they're different shapes of the same idea, a surface over weight space — and hit Run. The red ball is the current weight vector; the heat is the loss; the trail is the path taken.

The zigzag on "Elongated valley" is a real pathology — it's part of why the modern default isn't plain gradient descent but Adam, an optimizer that tracks per-weight step sizes. We'll get to that in the training post.

This is stochastic gradient descent: stochastic because each step uses a random mini-batch instead of the entire dataset. It is, mechanically, just a giant feedback loop. The error signal flows backward through the same computation graph that the input flowed forward through. The graph's state — the weights — drifts, over many iterations, toward a configuration that produces low loss across the training distribution.

In good distributed-systems terms: the parameter store is the system's only persistent state; the forward pass reads it; the backward pass writes to it; the dataset is the workload that defines what "correct" means. Once training ends, the weights are frozen, and the network becomes a pure function — same input, same output, every time.

Why does any of this work?

It's reasonable to ask: why should a stack of matrix multiplies and max(0, x)s be able to represent something like "is this picture a cat?"

There's a theoretical answer and a practical one.

The theoretical answer is the universal approximation theorem, proved independently by George Cybenko in 1989 and Kurt Hornik in 1991. It says, roughly: a neural network with one hidden layer and enough units, with a non-polynomial activation, can approximate any continuous function on a compact domain to arbitrary precision. With nonlinearities, neural networks are expressively general-purpose function approximators. The limit is not what they can represent; the limit is whether we can find weights that represent the function we want.

The practical answer is that for the kinds of functions we care about — pixels → labels, audio → text, text → text — the loss landscape, weird as it is in a billion dimensions, turns out to be navigable by gradient descent. Nobody predicted this would work as well as it has. It's an empirical fact, repeatedly confirmed since 2012, that big networks trained on big datasets with simple optimizers find good weights. Why exactly is still an active research question.

A 60-second history

The artificial neuron dates to McCulloch and Pitts (1943). Rosenblatt's perceptron in 1958 was the first machine that learned weights from examples. It was demonstrated in July 1958 at the U.S. Office of Naval Research, where an IBM 704 — a five-ton computer the size of a room — was fed a deck of fifty punch cards and, after 50 trials, taught itself to distinguish cards marked on the left from those marked on the right. Rosenblatt called it "the first machine which is capable of having an original idea."

Then Minsky and Papert published Perceptrons in 1969, proving that a single-layer perceptron couldn't compute XOR. Funding evaporated. The first AI winter set in.

The thaw came in 1986, when Rumelhart, Hinton, and Williams published "Learning representations by back-propagating errors" in Nature. Backprop made it tractable to train multi-layer networks. Yann LeCun's LeNet-5 in 1998 read postal zip codes for a living. But hardware and data weren't there yet for anything bigger.

Then in 2012, Krizhevsky, Sutskever, and Hinton entered AlexNet in the ImageNet competition and dropped the top-5 error rate from 26.2% to 15.3% in a single year. Geoffrey Hinton later put the shift bluntly: "by 2013, pretty much all the computer vision research had switched to neural nets." GPUs and ReLU, more than anything novel about the architecture, made it possible.

2017 brought the Transformer (Vaswani et al., "Attention Is All You Need"), which is the architecture under every modern LLM. We'll get to it later in this series.

What comes next

This post set up the substrate: a neural network is a parameterized function — billions of weights, organized into matrix-multiply layers with nonlinearities between them — fit to data by gradient descent on a loss. Everything else in modern AI is variations and elaborations on that.

The next posts in this series will dig into:

• What training actually means — pretraining a foundation model on a giant text corpus is a different beast from post-training (fine-tuning, RLHF) on curated examples. Both produce weights; they produce them very differently.
• Inference vs. training — what's actually happening when you hit a model with a prompt, and why it's so much cheaper than training.
• Why models hallucinate — a consequence of what we've just built, not a bug grafted on.
• Knowledge cutoffs — what it means that the weights are frozen.

---

---

Further reading

• Karpathy, A. (2017). "Software 2.0." — The framing this post borrows. Written when Karpathy was Director of AI at Tesla; still the clearest engineer-facing argument for why neural networks are a paradigm shift, not just a technique.
• Karpathy, A. "Neural Networks: Zero to Hero." — Build a tiny neural network library and a small GPT from scratch. The best hands-on complement to this post.
• Sanderson, G. (3Blue1Brown). "But what is a neural network?" — ~19-minute visual walkthrough. The best introduction that exists in video form.
• Nielsen, M. (2015). Neural Networks and Deep Learning. — Free online book. Chapter 4 ("A visual proof that neural nets can compute any function") is the most accessible treatment of universal approximation you'll find.
• Olah, C. (2014). "Neural Networks, Manifolds, and Topology." — A geometric way to see what a neural network is actually doing to its input. Beautifully written.
• Olah, C., Mordvintsev, A., & Schubert, L. (2017). "Feature Visualization." (Distill) — What individual neurons in a trained network actually respond to. The closest thing to looking inside a model.
• Quanta Magazine (2025). "AI Is Nothing Like a Brain, and That's OK." — Good antidote to the brain-metaphor confusion this post tries to head off.