LLM fundamentals for AI engineers
To build with LLMs, you need a working model of what happens between the text you send and the text you get back: text becomes tokens, tokens become vectors, a transformer predicts a probability for every possible next token, and one token is sampled at a time until the answer is done. Almost every practical question you will face, why it forgot something, why the answer changed, why it costs this much, why it made up a source, comes back to one of those steps. This section explains each of them at the level an AI engineer needs, without asking you to train a model.
Why AI engineers need the fundamentals
You can call an LLM API on your first day without knowing any of this, and many people do. The problems start when the application meets real users.
A chatbot forgets an instruction halfway through a long conversation. The same prompt gives a different answer on Tuesday than it did on Monday. The monthly bill is five times what you estimated. The model answers a question about your company confidently and wrongly. A user writes in Hindi and every request costs several times more than in English.
None of these are bugs in your code, and none of them are fixed by trying a different prompt at random. They are consequences of how LLMs work, and once you understand the mechanism, the fix is usually obvious.
What this section covers
The pages follow the path of one request through a model, and then the decisions you make around it.
| Page | The question it answers |
|---|---|
| How an LLM works | What happens between my prompt and the answer? |
| Tokens and tokenization | What does the model actually read, and why can't it count letters? |
| Context windows | How much can it see at once, and why does it forget? |
| Sampling | What do temperature, top-p and top-k change? |
| Nondeterminism | Why do I get different answers at temperature 0? |
| Embeddings | How does text become meaning a computer can compare? |
| Model families | Closed APIs, open-weight, small and reasoning models: what's the difference? |
| Choosing a model | How do I pick one for my product? |
| Pricing math | What will this cost per request, per user and per month? |
| Reasoning models | When is "thinking" worth the extra time and money? |
| Multimodal models | How do models work with images, audio and documents? |
| Structured outputs and function calling | How does a model return data and call tools? |
| Hallucination | Why does it make things up, and what actually reduces it? |
How to read this section
If you are new to LLMs, read the first four pages in order, because each one builds on the one before. Then open the rest when your build needs them: embeddings when you start retrieval, pricing before you estimate a project, function calling when you build your first agent.
If you already build with LLMs, jump to the pages that match the problems you are seeing. Most experienced developers find something new in the pages on nondeterminism, context windows and hallucination, because the common explanations of those three are incomplete.
What you don't need for this section
You don't need to know how to train a neural network, and you don't need calculus. Where math helps, like vectors and cosine similarity on the embeddings page or probabilities on the sampling page, the page explains exactly the part you need. If you want the math in more depth, the math page shows how much is needed and when.
From the floor
In one workshop on how LLMs work, we didn't write any code. We spent two hours on what each letter in GPT means, what a token is, what parameters hold, why the model doesn't learn from your chats, and how it predicts one word at a time, and the questions people asked, like whether a new chat remembers an old one or why the model doesn't get better as millions of people use it, went straight to these fundamentals.