A practical introduction to how modern LLMs work, from tokens and transformers to training and inference.
How tokenizers convert human language into numbers LLMs understand. Deep dive into Byte Pair Encoding (BPE), why character-level tokenization fails
