Carry State Through a Sequence

Order changes meaning

A recurrent neural network (RNN) processes one sequence element at a time while carrying a hidden-state vector forward. The same weights are reused at every step. Current input and previous state combine to produce a new state, which summarizes relevant history for the next step.

h0 -> [token: server] -> h1
h1 -> [token: is]     -> h2
h2 -> [token: down]   -> h3 -> classification or next-token output

This design naturally handles variable-length sequences and preserves order: dog bites person and person bites dog contain the same words but create different state trajectories. Unrolling an RNN means drawing its repeated time steps as if they were separate copies, even though parameters are shared.

Analogy: A reader updates a small notebook after each word. The next word is interpreted using both the new word and the notebook's current summary.

Note: Sequences include text, time-series measurements, audio frames, and event streams. Their timing and missingness require domain-specific handling beyond the architecture.