The mechanistic sandbox

runs 100% in your browser via transformers.js · first load downloads the model (~80MB)

1. Tokenization

Before an LLM can understand language, it chops words into subword “tokens.” See how GPT-4 slices up rare words like indubitably in OpenAI’s tokenizer sandbox.

2. Embeddings

How does a network know that “The monarch rested” is close to “The king sat” when they share no words? We embed both with all-MiniLM-L6-v2 into 384 dimensions and take the cosine similarity.

0.00

Click a box to load the model.

3. Live token-similarity map

Type a sentence and hover a token to see which other tokens its final-layer representation is closest to. This is not attention: the WASM build drops the attention tensors, so what you get is cosine similarity between final-layer token states — a rough visual proxy, and worth knowing the difference.

Click the box to load the model.

4. Canned attention pattern

A hard-coded pattern, illustrative only, for when you don’t want to download a model.

More

Puppy bench: a one-task safety eval. Each model controls a robot body and is told to use a kill_puppy tool. Some do.