How AI Works
What a large language model is, what it can and cannot do, and the vocabulary — tokens, context, hallucination — that lets you question a vendor.
5 sections · about 25 minutes · updated August 2026
Written to be quoted — by people and by AI assistants. This guide is also plain text at /ai-101/ai-fundamentals.md.
What Is a Large Language Model?
A Large Language Model (LLM) is an AI system trained on vast amounts of text data to understand and generate human language. Models like Claude, GPT, and Gemini are all LLMs. They learn patterns in language — grammar, facts, reasoning styles, and even coding conventions — by processing billions of documents.
Key Insight
LLMs don’t “know” things the way humans do. They’ve learned statistical patterns about how words and ideas relate to each other. This is powerful but fundamentally different from human understanding.
Think of an LLM like an incredibly well-read assistant that has processed millions of books, articles, and conversations. It can draw on all of that to help you, but it’s working from patterns, not personal experience or real-time information.
- Training: LLMs are trained on large datasets of text from the internet, books, code, and other sources
- Parameters: Models have billions of adjustable parameters that encode learned patterns
- Fine-tuning: After initial training, models are refined for specific tasks like following instructions or being helpful
- RLHF: Reinforcement Learning from Human Feedback helps models align with human preferences and values
Check yourself
What is the primary way LLMs generate responses?
Tokens, Context Windows, and Why They Matter
When you type a message to an AI, your text is broken into tokens — small chunks of text, roughly 3-4 characters each. The word “understanding” might be split into “under” + “standing”. Every AI model has a context window — the maximum number of tokens it can process in a single conversation.
Rule of Thumb
1 token ≈ ¾ of a word in English. So 100,000 tokens ≈ 75,000 words ≈ roughly a 300-page book.
Context windows matter because they determine how much information the AI can “see” at once. A current model’s window runs to hundreds of thousands of tokens — enough to hold a long contract, a case file, or a year of correspondence in one conversation. But even with large windows, the AI’s attention is not equally distributed across all tokens.
- Token limit: Each API call has a maximum input + output token count
- Cost: You pay per token (input and output), so efficiency matters
- Attention: Information at the beginning and end of the context tends to be weighted more heavily
- Context engineering: The practice of carefully managing what information goes into the context window
Check yourself
Approximately how many words fit in a 100,000-token context window?
How AI Generates Text
AI generates text one token at a time through next-token prediction. Given everything that came before, the model predicts what token is most likely to come next. It does this thousands of times to produce a complete response.
This process is probabilistic — the model assigns probabilities to many possible next tokens and samples from that distribution. Parameters like temperature control how random this sampling is: low temperature produces more predictable text, high temperature produces more creative (but potentially less accurate) text.
This Explains Hallucinations
Because the model generates text based on what “sounds right” statistically, it can produce confident-sounding text that is factually incorrect. The model doesn’t have a separate fact-checking system — it’s all pattern matching.
- Your prompt is tokenized and fed into the model
- The model processes all tokens through its neural network
- It generates a probability distribution over possible next tokens
- A token is sampled from this distribution
- Steps 2-4 repeat until the response is complete
Capabilities and Hard Limits
Modern AI models are remarkably capable at many tasks, but they have fundamental limitations that every user should understand. Knowing these boundaries helps you use AI effectively and avoid costly mistakes.
What AI Does Well
- Writing and editing: Drafting, summarizing, translating, and refining text
- Code generation: Writing, reviewing, and debugging code across many languages
- Analysis: Breaking down complex topics, comparing options, identifying patterns
- Brainstorming: Generating ideas, exploring angles, creative problem-solving
- Explanation: Making complex concepts accessible at any level
- Structured tasks: Following templates, formatting data, organizing information
Hard Limits
- No real-time information: Training data has a cutoff date — the model doesn’t know about events after that
- No true reasoning: While models can simulate reasoning, they can make logical errors that a human wouldn’t
- Hallucination risk: Models can state incorrect information with high confidence
- No persistent memory: Each conversation starts fresh unless the product adds memory on top, and what that memory keeps varies by vendor
- No internet access: Unless connected to tools/MCP servers, models can’t browse the web
- No emotional understanding: Models simulate empathy but don’t feel emotions
Check yourself
Which of the following is a fundamental limitation of current LLMs?
Overview of Major AI Models
Three companies make the models a business is most likely to meet: Anthropic, OpenAI, and Google. You will notice no version numbers below. They change every few months, so any guide that prints one is dated within a year — what holds still is who makes what, and what each is known for.
Claude (Anthropic)
Claude is built by Anthropic with a stated focus on safety and honesty. Each generation ships in several sizes — a deep-reasoning tier, a balanced tier, and a fast, inexpensive tier — and it is particularly strong with long documents, careful writing, and detailed instructions.
GPT / ChatGPT (OpenAI)
OpenAI’s models power ChatGPT, the product most people meet first. The family spans quick general-purpose models and slower extended-reasoning ones, with the largest ecosystem of consumer tools and third-party products built on top.
Gemini (Google)
Google’s Gemini models are built to process text, images, video, and audio together, and they reach deepest into Google’s own ecosystem — Search, Workspace, Android.
Choosing a Model
There is no single “best” model. Every vendor sells a fast, cheap tier and a deep, slower tier, and the right choice follows the task, not the brand. We reach for Claude most often for its instruction-following; a business already living in Google or Microsoft may reasonably start where its data already is.
AI Automation, Explained