How AI Works
What a large language model is, what it can and cannot do, and the vocabulary that lets you question a vendor: tokens, context, hallucination.
- 5 sections
- about 8 minutes
- updated October 2026
Written to be quoted, by people and by AI assistants. This guide is also plain text at /ai-101/ai-fundamentals.md.
What Is a Large Language Model?
A Large Language Model (LLM) is an AI system trained on vast amounts of text data to understand and generate human language. Models like Claude, GPT, and Gemini are all LLMs. They learn patterns in language by processing billions of documents: grammar, facts, reasoning styles, and even coding conventions.
Key Insight
LLMs don’t “know” things the way humans do. They’ve learned statistical patterns about how words and ideas relate to each other. That is useful, and it is not the same as understanding.
Think of an LLM as a very well-read assistant that has processed millions of books, articles, and conversations. It can draw on all of that to help you, but it works from patterns, not personal experience or real-time information.
- Training: LLMs are trained on large datasets of text from the internet, books, code, and other sources
- Parameters: Models have billions of adjustable parameters that encode learned patterns
- Fine-tuning: After initial training, models are refined for specific tasks like following instructions or being helpful
- RLHF: Reinforcement Learning from Human Feedback helps models align with human preferences and values
Tokens, Context Windows, and Why They Matter
When you type a message to an AI, your text is broken into tokens: small chunks of text, roughly 3-4 characters each. The word “understanding” might be split into “under” + “standing”. Every AI model has a context window: the maximum number of tokens it can process in a single conversation.
Rule of Thumb
1 token ≈ ¾ of a word in English. So 100,000 tokens ≈ 75,000 words ≈ roughly a 300-page book.
Context windows matter because they determine how much information the AI can “see” at once. A current model’s window runs to hundreds of thousands of tokens, enough to hold a long contract, a case file, or a year of correspondence in one conversation. But even with large windows, the AI’s attention is not equally distributed across all tokens.
Try it
Tokens and the context window
A model reads text as tokens, and its context window holds a fixed number of them. When a conversation outgrows the window, something has to go. Usually it is the oldest part, and the model no longer sees it.
Type in the box. Watch the window fill, then watch the oldest tokens fall out.
- Ple (fell out)
- ase (fell out)
- send (fell out)
- the (fell out)
- sig
- ned
- engag
- ement
- let
- ter
- back
- bef
- ore
- Fri
- day
- .
In the window 12Fell out 4Empty places 0
A simplified illustration. Real tokenizers split text differently, and a real window holds hundreds of thousands of tokens.
- Token limit: Each API call has a maximum input + output token count
- Cost: You pay per token (input and output), so efficiency matters
- Attention: Information at the beginning and end of the context tends to be weighted more heavily
- Context engineering: The practice of carefully managing what information goes into the context window
Check yourself
Approximately how many words fit in a 100,000-token context window?
How AI Generates Text
AI generates text one token at a time through next-token prediction. Given everything that came before, the model predicts what token is most likely to come next. It does this thousands of times to produce a complete response.
This process is probabilistic: the model assigns probabilities to many possible next tokens and samples from that distribution. Parameters like temperature control how random this sampling is: low temperature produces more predictable text, high temperature produces more creative (but potentially less accurate) text.
Try it
How a model picks the next word
A model does not look up the next word. It gives every candidate a chance and draws one at random, weighted by those chances. Temperature reshapes the odds: at a low temperature the favorite is nearly certain, and at a high one the unlikely words get a real chance.
Move the temperature. Then pick a word a few times and see what comes out.
The prompt
Our office is open Monday through ___
- Friday63%
- Saturday16%
- Thursday12%
- Sunday6%
- Wednesday3%
- lunch1%
One draw
A pick is a random point on this line. The wider a word’s stretch, the more often the point lands on it.
Picked so far: Nothing yet.
Every one of these words sounds right. Only one can be true of your office.
These numbers are an illustration, not measurements of any real model. The formula is the real one: each word’s score is divided by the temperature, then the scores are turned into chances that add up to 100 percent.
This Explains Hallucinations
Because the model generates text based on what “sounds right” statistically, it can produce confident-sounding text that is factually incorrect. The model doesn’t have a separate fact-checking system. It’s all pattern matching.
- Your prompt is tokenized and fed into the model
- The model processes all tokens through its neural network
- It generates a probability distribution over possible next tokens
- A token is sampled from this distribution
- Steps 2-4 repeat until the response is complete
Check yourself
What is the primary way LLMs generate responses?
Capabilities and Hard Limits
AI models do many tasks well, and they have hard limits every user should understand. Knowing the limits helps you use AI well and avoid costly mistakes.
What AI Does Well
- Writing and editing: Drafting, summarizing, translating, and refining text
- Code generation: Writing, reviewing, and debugging code across many languages
- Analysis: Breaking down complex topics, comparing options, identifying patterns
- Brainstorming: Generating ideas, exploring angles, creative problem-solving
- Explanation: Making complex concepts accessible at any level
- Structured tasks: Following templates, formatting data, organizing information
Hard Limits
- Training cutoff: answers come from training data unless the product searches the web. Check which it did
- Logic errors: models can make reasoning mistakes a person would catch, so check anything that matters
- Hallucination risk: Models can state incorrect information with high confidence
- No persistent memory: Each conversation starts fresh unless the product adds memory on top, and what that memory keeps varies by vendor
- No emotional understanding: Models simulate empathy but don’t feel emotions
Check yourself
Which of the following is a fundamental limitation of current LLMs?
Overview of Major AI Models
Three companies make the models a business is most likely to meet: Anthropic, OpenAI, and Google. You will notice no version numbers below. They change every few months, so any guide that prints one is dated within a year. What holds still is who makes what, and what each is known for.
Claude (Anthropic)
Claude is built by Anthropic with a stated focus on safety and honesty. Each generation ships in several sizes: a deep-reasoning tier, a balanced tier, and a fast, inexpensive tier. Claude is particularly strong with long documents, careful writing, and detailed instructions.
GPT / ChatGPT (OpenAI)
OpenAI’s models power ChatGPT, the product most people meet first. The family spans quick general-purpose models and slower extended-reasoning ones, with the largest ecosystem of consumer tools and third-party products built on top.
Gemini (Google)
Google’s Gemini models are built to process text, images, video, and audio together, and they reach deepest into Google’s own ecosystem: Search, Workspace, Android.
Choosing a Model
There is no single “best” model. Every vendor sells a fast, cheap tier and a deep, slower tier, and the right choice follows the task, not the brand. We reach for Claude most often for its instruction-following; a business already living in Google or Microsoft may reasonably start where its data already is.
AI Automation, Explained