tokenbricks

Paste text, pick a tokenizer, and it's BPE-encoded right there in your browser — the exact byte-pair encoding OpenAI's models use, via gpt-tokenizer. Below it: the same text redrawn as a wall of equal-width bricks, one per token, so you can see the text the way the model actually experiences it — by token count, not character count. A short word can be one cheap token; a whole rare word can also be one. Inspired by tiktokenizer, prompted by astrra.space's idea for qustrolabe.bsky.social.

tokenizer

text

Method: text is BPE-encoded client-side with gpt-tokenizer (loaded as a plain ES module from esm.sh, no wasm, no model weights) — the same algorithm and vocabularies OpenAI's tokenizers use, so token boundaries here match the real API. Each resulting token id is decoded back to text on its own to draw its brick; a handful of tokens are single bytes of a multi-byte character and can't stand alone, so they may render as a stray glyph or an empty dashed brick — that's the tokenizer being honest about how the text was actually cut, not a bug. Nothing you type is sent anywhere; everything above runs in this tab.