Supervised vs Unsupervised Learning: Worked Examples

In brief

Supervised learning trains on examples that carry the right answer, and unsupervised learning trains on data that has none. IBM and Google Cloud both draw the line there. A chatbot like InstructGPT used neither alone: it began as GPT-3, trained on raw text, then got supervised and reinforcement tuning, according to OpenAI's 2020 and 2022 papers.

What is the difference between supervised and unsupervised learning?

The difference is whether each training example comes with the answer. Google Cloud puts it in one line: "Supervised learning uses labeled training data, and unsupervised learning does not." Everything else follows from that.

Two shallow black trays on a black plinth beside a brass abacus: in one tray glass spheres each resting on a small brass tag, in the other the same spheres in rainbow glass loosely grouped into two clusters with no tags, in a charcoal studio under white rim light.
Supervised learningUnsupervised learning
Training dataInputs paired with the right output, called labelsInputs only, no labels
GoalPredict the label for a new inputFind groups, patterns or odd cases
Typical tasksClassification (spam or not), regression (a price)Clustering, association, dimensionality reduction
Example methodsSupport vector machines, random forests, linear regressionK means, autoencoders, principal component analysis
How you score itCompare predictions with held back answersA person reads the groups and judges them
Main costSomeone has to label the dataSomeone has to decide whether the output means anything
Main riskLearns the labelers' mistakesFinds a pattern that isn't useful

IBM's explainer names the task types and most of the methods: support vector machines, linear regression, random forest, k means and autoencoders. Principal component analysis is the usual example for dimensionality reduction. Both the table and the rest of this page sit under what is machine learning, which covers the wider field.

What is supervised learning, with a tiny example?

Supervised learning is learning from solved examples. You show the model inputs together with the right answer, it finds the rule that links them, and you then give it inputs it hasn't seen. The answer can be a category, which is classification, or a number, which is regression.

Here is a six email dataset written for this page. Each email has two counts and a label.

EmailLinksALL CAPS wordsLabel
100Not spam
210Not spam
301Not spam
454Spam
576Spam
645Spam

The simplest model learns the average email of each label. Spam averages 5.3 links and 5 caps words. Not spam averages 0.3 links and 0.3 caps words. A new email arrives with 6 links and 3 caps words. Its straight line distance to the spam average is 2.1 and to the not spam average 6.3, so the model says spam.

That is a nearest average classifier, chosen because you can check it by hand with a pencil and a calculator in about two minutes. Real filters use thousands of features and methods such as the random forests IBM names. The loop is the same: labeled rows in, a rule out, a prediction on new rows.

What is unsupervised learning, on the same six emails?

Unsupervised learning is learning with no answers. The model gets the inputs and has to find structure by itself, usually groups. Google Cloud says these algorithms "work independently to learn the data's inherent structure."

Take the same six emails and delete the Label column. Now run k means with two groups, the method IBM names for clustering. It starts with two guess centers, here emails 1 and 4, and repeats two steps.

  1. Assign each email to the nearest center. Emails 1, 2 and 3 go to center A at (0, 0). Emails 4, 5 and 6 go to center B at (5, 4).
  2. Move each center to the average of its emails. A moves to (0.3, 0.3). B moves to (5.3, 5).

A second pass moves no email. It stops. The result is two groups, called only group A and group B. The model never says spam, because no one gave it that word. A person looks at group B, sees many links and many caps words, and names it.

The groups matched the labels here because the data was clean on purpose. Real data is messier, and that is why unsupervised output needs a person to read it. IBM warns that it can give "wildly inaccurate results" without that check.

What is semi supervised learning?

Semi supervised learning trains on a small labeled set and a large unlabeled one. It fits when labels cost money and raw data is cheap. Google Cloud describes it as training on a small labeled set, then labeling the larger set with the model's own guesses, called pseudo labels, and repeating.

Back to the six emails. Say only email 1 (not spam) and email 4 (spam) carry labels. Email 2 sits 1 away from email 1 and about 5.7 from email 4, so it gets the not spam pseudo label. Email 3 follows it. Emails 5 and 6 sit closer to email 4, so they become spam. All six rows now have a label, and you train a supervised model on them.

The risk is plain. A wrong pseudo label goes back into training, so the next round learns from the mistake and can repeat it across thousands of rows before anyone looks at a single one. IBM's example is a radiologist who labels a small set of CT scans for tumors, and a model handles the rest.

What are reinforcement and self supervised learning?

Reinforcement learning learns from rewards, not answers. IBM defines it as a process where "autonomous agents learn to make decisions by interacting with their environment." The agent tries an action, gets a reward or a penalty, and aims to collect as much reward as it can over time. Game players and robots are the classic cases.

Self supervised learning makes its own labels from the data. IBM says these models "generate implicit labels from unstructured data." For language, it hides a word or takes the next word in a sentence, and the real word is the answer. No person labels anything, yet the model gets millions of solved examples. IBM calls it a subset of unsupervised learning that also borrows the loss function of supervised learning, which is why it gets its own category.

Is ChatGPT supervised or unsupervised?

Neither one alone. OpenAI's papers show a chatbot built in stages, and each stage uses a different type of learning.

StageLearning typeWhere the answer comes fromSource
Pretraining GPT-3Self supervisedThe next word in the text itselfBrown et al., May 2020
Instruction tuningSupervisedLabeler demonstrations of good repliesOuyang et al., March 2022
Preference tuningReinforcementHuman rankings of model outputsOuyang et al., March 2022

GPT-3 was an "autoregressive language model with 175 billion parameters," and the InstructGPT team fine tuned it first with supervised learning on labeler demonstrations, then with reinforcement learning from human feedback. The payoff, in their words: "outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3."

So a model 100 times smaller won on human preference once it got the supervised and reward stages. Those papers cover GPT-3 era models, and later models from OpenAI and others use their own recipes, so check each lab's model card. What is fine tuning and what is an LLM cover the stages in more depth.

Which real problems fit each type?

Three cases from the pages Google ranks show the split.

  1. Predicting flight times from weather and airport traffic is supervised. Google Cloud gives it as its example, and past flights supply the answers.
  2. Grouping shoppers who buy related products is unsupervised. Google Cloud and IBM both use it, and no one knows the groups beforehand.
  3. Spotting tumors on CT scans with only a few labeled is semi supervised, IBM's case, because experts are scarce and scans are plentiful.

Fraud shows how the choice depends on the data. If you hold past transactions marked fraud or fine, a supervised classifier learns what fraud looked like. If you hold no such labels, anomaly detection flags transactions that look unlike the rest, which is unsupervised. One goal, two methods, picked by what you have.

Stock prices are the opposite trap. One top result lists price prediction as a plain supervised task. The method fits, since past prices are labels, but the signal is noisy and a model that fits last year can fail next year. A task suiting a method doesn't mean the method will work.

How else do supervised and unsupervised learning differ?

Beyond labels, IBM lists a few trade offs, and they are IBM's own claims rather than measured benchmarks. It describes supervised models as generally more accurate but dependent on people to label data first, and as slow to train on large data sets. It describes unsupervised learning as computationally complex, needing large training sets, and prone to inaccurate output without human review.

Patsnap Eureka adds a business view: segmentation helps personalization, while supervised models fit automated decisions. Its supervised cases are sales forecasts, customer churn, house prices and disease diagnosis. Its unsupervised cases are market segmentation, machine fault detection, social network analysis and image compression.

When should you use supervised or unsupervised learning?

Use supervised learning when you can name the answer and have enough labeled examples. Use unsupervised learning when you want to find out what is in the data. Use semi supervised when labels are costly, and reinforcement when each choice changes what comes next.

Your situationUseWhy
You have past rows with the right answerSupervisedThe model can be scored against held back answers
You have data and no idea what groups existUnsupervisedClustering finds groups without labels
A few labels, a lot of raw dataSemi supervisedPseudo labels stretch the small set
A huge pile of text or images, no labelsSelf supervisedThe data supplies its own targets
A sequence of choices with a score at the endReinforcementReward steers each step

Two checks before you commit. First, ask what a wrong answer costs, because supervised models are easier to score and unsupervised ones lean on human judgment. Second, ask who labels the data. If the answer is an expert working slowly, plan for the cost before you plan the model.

What questions can you ask an AI about this?

An AI chat works well as a tutor on this topic, because you can hand it your own data. Paste these into Claude, ChatGPT, Gemini or Whizi and fill the brackets.

  1. "I have [describe your data, such as 800 support tickets with a category column]. Which type of learning fits, and what is the first method to try?"
  2. "Here are six rows: [paste]. Cluster them by hand with k means for two groups, and show each step so I can check it."
  3. "Give me a quiz of five short cases. For each, I say supervised, unsupervised, semi supervised, self supervised or reinforcement, and you tell me why I was right or wrong."
  4. "My model got [accuracy] on labeled data and fails on new data. List three likely causes in order, and one test for each."
  5. "I want to group [customers, documents or images]. How would I check that the groups mean something, without labels?"
  6. "Explain how a chatbot goes from raw text to following instructions, using the words supervised, self supervised and reinforcement."

Ask for the working, not just the label. Don't accept a bare answer. A model that shows its steps on prompt 2 gives you something to check.

Where does Whizi fit when you study this?

Whizi is one app instead of a pile of single purpose tools, and it suits this topic because you can ask one question and get two views. Web search is on by default in each chat, so a model can pull in a current source.

The free plan gives you 10 messages a day on Whizi AI, Auto and Claude Haiku, with no card. Pro carries Claude, GPT and Gemini in one app, and its first week is $1.99. On Pro, side by side model chats send prompt 2 to two models, so you read only where their k means steps disagree.

A textbook or a free course is still better for a full path from zero. Whizi helps once you have a question about your own data.

What can an AI get wrong about these terms?

AI models explain the three main types well, and they blur the edges. Self supervised learning is the usual blur: some sources file it under unsupervised learning and others give it its own label. IBM does both, calling it a subset of unsupervised learning with its own category. Ask the model which source it follows.

Hand worked numbers need a check too. A model can add the straight line distances wrong, so redo one step of any k means answer yourself. For a school or work claim, open the page the number came from, as the fact check routine shows.

Workflow checklist
  • Ask first: does each training row carry the right answer?
  • Count your labeled rows and price one label before choosing supervised learning
  • Read every unsupervised group yourself before you name it
  • Treat pseudo labels as guesses and spot check a few
  • Redo one hand worked step of any AI explanation
Common questions

Frequently asked questions

Is clustering supervised or unsupervised?

Clustering is unsupervised learning, because it groups data with no labels. IBM lists it beside association and dimensionality reduction as the three main unsupervised tasks, and k means is its example method.

What is the difference between semi supervised and self supervised learning?

Semi supervised learning needs a small set of human labels and guesses the rest, while self supervised learning needs none because the data supplies its own targets, such as the next word in a sentence. Google Cloud and IBM cover the first, and IBM has a separate page on the second, where it calls the method a subset of unsupervised learning.

Is supervised learning more accurate than unsupervised learning?

They answer different questions, so accuracy doesn't compare directly. IBM calls supervised models generally more accurate, but a supervised model is scored against known answers and an unsupervised one has no answer key, so a person judges whether its groups are useful.