Eye of the Storm

Retrieval-augmented generation is one hard problem and one trivial one wearing the same name. The generation half is a prompt with text pasted into it. Everything that actually breaks is a search problem that predates LLMs by thirty years — which is why so many teams staff it wrong.

Tipping, threatening, and 'let's think step by step' mostly failed replication or got absorbed into the models. What survived is less glamorous and more useful: prompting is steering a frozen distribution, and the only thing that turns it into a discipline is measuring.

Transformers didn't win because attention beats recurrence — they won because attention is parallelizable at training time, which made them GPU-native and unlocked the scaling laws. Why the moat is the stack, not the architecture, and who's actually challenging it.

An embedding turns meaning into geometry — text mapped to a vector so that similar meanings land close together. The two very different artifacts both called 'embeddings,' how sentence models are actually trained, and what vectors can and can't tell you.