Transformers

Relatively short post today. I came across this amazing tool called Transformer Explainer. It lets you see how a Transformer model (like GPT-2) actually works, layer by layer, right in your browser. You can watch tokens flow through embeddings, self-attention, and MLPs, and even play around with parameters like temperature and top-k to see how text generation changes. Check it out here: https://poloclub.github.io/transformer-explainer/ Written for KiwiGPT.co.nz — Generated, Published and Tinkered with AI by a Kiwi ...

In the world of AI, context isn’t just helpful—it’s everything. Whether it’s the billions of tokens in a language model’s attention mechanism, the rapidly shifting mosaic of user intent, or the structured metadata keeping facts grounded, context defines what AI can do. 1. A Foundation Built on Attention The 2017 paper “Attention Is All You Need” introduced the Transformer model, which revolutionized deep learning by letting models process all parts of a sequence in parallel using self-attention. No more recurrences—just powerful, context-aware computation. Since then, it’s become the backbone of modern models like GPT, BERT, and beyond. 1 ...