AI Basics

What a neural network actually is

A neural network is a function with billions of tunable weights fit from examples by an optimizer — not a brain simulation. A ground-up explainer for software engineers on how they actually work.

What training actually means

Training an LLM isn't one step — it's a pipeline. Pretraining (99% of the compute) gives the model its knowledge; post-training (SFT, RLHF) shapes how it behaves. A ground-up map for engineers.

Inference vs. training

Once training ends, the weights freeze. Inference is the read-path — a forward pass run one token at a time, forever. Autoregressive decoding, prefill vs. decode, the KV cache, and the five ways inference differs from training.

Why models hallucinate

Hallucination isn't a bug grafted onto a truthful system — it's the default behavior of a next-token predictor. The same operation produces every correct answer and every confident fabrication. Why it happens, and why grading rewards it.

Knowledge cutoffs: the model thinks it's still last year

A knowledge cutoff isn't a date on a sticker — it's the trailing edge of a distribution, and the model can't reliably see its own boundary. Why frozen weights make every chat a conversation with an archive, and what actually moves the line.

Tokens, Tokenizers, and How Claude Counts Them

How subword tokenization algorithms work under the hood, why they shape context window economics and model accuracy, and what Claude's proprietary tokenizer changes mean for your API budget.

Context windows: the only door into a frozen model

A frozen model has exactly one write surface — the context window. Why it's bounded, why its usable length is far shorter than the number on the model card, and why deciding what to put in it is the core skill of working with LLMs.

Roles are a convention, not an architecture

System, user, and assistant are special tokens in one flat sequence that the model was post-trained to honor — not privileged channels. Why the instruction hierarchy is a learned preference, and why that's exactly why prompt injection works.

Temperature and sampling: the dice live outside the model

The model emits a fixed probability distribution; everything that feels random happens in the sampler on top of it. What temperature actually does, how top-k/top-p/min-p differ, and why 'temperature 0' isn't truly deterministic.

Streaming: publishing, not generating

Streaming doesn't change how a model generates—only how the server delivers what it was already producing one token at a time. What SSE actually is, why TTFT and ITL are different problems, and the engineering a stream forces on you.

Structured output: making a text generator return typed data

Structured output is constrained decoding: before the sampler picks, a schema-compiled state machine masks every token that would break your JSON, making invalid shape mechanically impossible. Three tiers of guarantee, why tool calls are the same machinery, and the quality tradeoff nobody warns you about.