Playground

Type any sentence and watch it tokenized, then see a real attention computation run on it live: the same math as the Attention chapter, free to run on your own words.

Type any sentence

Tokens: 8Characters: 30

Select a query token

Live attention weights from "The"

The

patie

##nt

fell

near

the

bed

This is real softmax(QKᵀ/√d) attention, computed live for whatever you type, using deterministic, untrained stand-in vectors for Q and K (the same kind used in the Query · Key · Value chapter). Because the vectors aren't trained, the pattern won't be linguistically meaningful; what's real here is the computation, not the weights. Here every token can see every other token; GPT-style generators hide later tokens with a causal mask (see the Attention chapter).