What you'll learn
Until now our models labelled the world. Generative AI flips that: it creates — text, images, audio, code. This short module frames the shift and previews the three great families of generative models before we dive into each.
By the end of this module you'll be able to:
- Distinguish discriminative from generative models
- Explain what it means to model a distribution
- Name the major generative families
- Think clearly about machine "creativity"
Discriminative vs generative
A discriminative model learns to tell classes apart — the decision boundary. A generative model learns what the data itself looks like, so thoroughly that it can produce brand-new examples:
Discriminative
Learns the boundary between classes.
asks: "is this a cat or a dog?"
models P(label | data)
Everything in Phases 2–6 so far.
Generative
Learns what the data itself looks like.
asks: "what does a cat look like?"
models P(data) — so it can create more
LLMs, diffusion, GANs.
Modeling a distribution
Generation means learning the probability distribution of the data — which combinations of pixels look like real photos, which sequences of words read like real English. Once a model captures that distribution, it can sample from it: draw a new point that's plausible but never seen before. That sample is the generated image, sentence, or song.
Key idea
The generative families
Three architectures dominate generative AI, each suited to different data:
| Family | How it generates | Best known for |
|---|---|---|
| LLMs (Transformers) | Predict the next token, over and over | Text, code, chat |
| Diffusion models | Start from noise, denoise step by step | Images, video, audio |
| GANs | A generator vs a discriminator, competing | Sharp images, faces |
What "creativity" means here
Is the model being creative? It isn't copying — a good generator produces genuinely new samples. But it also isn't conjuring from nothing: it recombines patterns learned from its training data. That's a powerful and useful kind of novelty, and also the source of real questions about originality, copyright, and bias that we return to in Module 31.
Recap & quick check
Key takeaways
- Discriminative models label data (P(label | data)); generative models create data (P(data)).
- Generation works by learning the data's probability distribution and sampling from it.
- The three main families are LLMs (next-token), diffusion (denoising), and GANs (adversarial).
- Generated outputs are genuinely new but recombine patterns from training data.
- This raises real questions of originality, copyright, and bias (Module 31).
Quick check
1. What does a generative model learn?
2. A discriminative model answers the question…
3. Which family generates images by denoising from random static?
4. Is a generative model simply copying its training data?
Let's open up the most influential generative model of all — the large language model. Next up: Module 26 — Large Language Models.