Inside the LLM
An interactive study of how language models turn text into representations, computations, and predictions.
Developed by Fransiskus Hutagalung || Computer Science
What is this?
Inside the LLM is an educational project built to make the internal mechanics of language models easier to inspect. It focuses on the steps between an input sentence and a predicted next token: tokenization, embeddings, attention, transformer blocks, logits, probability distributions, and autoregressive generation. Instead of treating the model as a black box, the project exposes the intermediate representations and computations that can be shown meaningfully in an interactive interface.
What will you learn?
Beyond the transformer
A transformer forward pass is only one part of a modern language system. Real applications can add retrieval, external tools, conversation context, memory, post-training, and other system components around the underlying model. Products such as ChatGPT, Claude, and Gemini therefore involve more than the transformer architecture itself. This project focuses on the model-side concepts that can be explained and demonstrated directly.
The visualizations are educational representations of the underlying concepts. They simplify some implementation details so that the calculations and relationships remain visible. They are not intended to reproduce the proprietary internal implementation of any specific commercial model.
Principle
The project follows a simple idea: if a concept can be calculated, its calculation should be possible to inspect. Explanations provide the context; visualizations show the structure; interaction lets you test what happens when the input changes.
About the developer
This project was developed by Fransiskus Hutagalung as a personal Computer Science project focused on making machine learning and language-model concepts easier to study through direct interaction. The goal is not to hide the mathematics behind a polished interface, but to use the interface as a way to inspect the mathematics, representations, and transformations involved in a language model.