Deep Learning CourseLesson 1 of 16 17 min

Your First Deep Learning Model

Deepak Mishra, Senior Technology Lead at Incedo

What you'll learn: how to train a deep learning model that recognises 37 breeds of cats and dogs, in about 30 lines of code, and then use it on your own photos. Along the way you'll see what the model actually outputs, why those outputs are probabilities, and what makes a set of numbers a probability distribution.

Before you start: basic Python (lists, functions, loops). No machine learning or maths background is needed. We use a free GPU in Google Colab.

Get a free GPU

Open a new notebook in Google Colab, then choose Runtime > Change runtime type > T4 GPU. Deep learning runs on graphics cards because training is millions of multiplications that can happen in parallel, which is exactly what a GPU is built for. Check that PyTorch can see it:

Python

import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
print("Using:", device)   # 'cuda' means the GPU is ready

Train a pet-breed classifier

Run this cell. It downloads the Oxford-IIIT Pet dataset (about 7,400 photos of 37 breeds), takes a network that has already learned to see from 1.2 million photos, and teaches it the breeds. On a Colab GPU it takes a few minutes:

Python

from torch import nn
from torchvision import datasets, models, transforms

weights = models.ResNet18_Weights.DEFAULT
preprocess = weights.transforms()      # resize, crop and normalise photos the way the model expects

train_data = datasets.OxfordIIITPet("data", split="trainval", download=True, transform=preprocess)
test_data = datasets.OxfordIIITPet("data", split="test", download=True, transform=preprocess)
train_loader = torch.utils.data.DataLoader(train_data, batch_size=64, shuffle=True, num_workers=2)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=64, num_workers=2)

model = models.resnet18(weights=weights)       # a network already trained on ImageNet
for param in model.parameters():
    param.requires_grad = False                # freeze everything it already knows
model.fc = nn.Linear(model.fc.in_features, 37) # a new final layer: one output per breed
model = model.to(device)

loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.fc.parameters(), lr=1e-3)

for epoch in range(3):
    model.train()
    for images, labels in train_loader:
        images, labels = images.to(device), labels.to(device)
        loss = loss_fn(model(images), labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    model.eval()
    correct = 0
    with torch.no_grad():
        for images, labels in test_loader:
            predictions = model(images.to(device)).argmax(dim=1)
            correct += (predictions == labels.to(device)).sum().item()
    print(f"epoch {epoch + 1}: test accuracy {correct / len(test_data):.1%}")

Output

epoch 1: test accuracy 78.7%
epoch 2: test accuracy 84.9%
epoch 3: test accuracy 87.4%

After three passes over the data, the model typically identifies the breed of around 85 to 90% of photos it has never seen. Random guessing among 37 breeds would score under 3%. Many breeds are hard even for people: try telling an American Pit Bull Terrier from a Staffordshire Bull Terrier.

Try it on your own photo

Upload a photo of a cat or dog and ask the model what it sees. The model returns a score for every breed; we turn the scores into probabilities and print the top three:

Python

from google.colab import files
from PIL import Image

uploaded = files.upload()                       # pick a photo from your computer
photo = Image.open(next(iter(uploaded))).convert("RGB")

model.eval()
with torch.no_grad():
    scores = model(preprocess(photo).unsqueeze(0).to(device))[0]   # 37 raw scores
    probabilities = torch.softmax(scores, dim=0)

top = probabilities.topk(3)
for prob, index in zip(top.values, top.indices):
    print(f"{train_data.classes[index]:30s} {prob.item():.1%}")

What just happened

You didn't teach a network to see from scratch. Training a large image network from nothing takes millions of labelled photos and days of computing. Instead you used transfer learning. ResNet-18 had already been trained on ImageNet, a collection of 1.2 million photos in 1,000 categories. Along the way, its layers learned general visual features: edges, textures, fur patterns, eyes, ears, snouts. Those features are useful for almost any picture.

You kept all of that, froze it so it wouldn't change, and replaced only the network's last layer, the one that turns features into answers, with a fresh layer for 37 breeds. Training then only had to learn how to combine features the network already knew into breed decisions. That is why it worked with a few thousand photos and a few minutes.

The training loop itself is four lines that you'll meet in every lesson: compute the loss (how wrong the model is), clear the old gradients, backpropagate (work out how to change each weight), and step (change them a little). Lesson 7 takes those four lines apart completely. Here we look at something simpler: what goes in, and what comes out.

What goes in: a photo is a grid of numbers

To the model, a colour photo is three grids of numbers, one each for red, green and blue, each number saying how bright that channel is at that pixel. The preprocessing step resizes every photo to 224 × 224 pixels, so each one becomes a tensor (a multi-dimensional array) of 3 × 224 × 224 = 150,528 numbers:

Python

image, label = train_data[0]
print(image.shape)                       # torch.Size([3, 224, 224]): channels, height, width
print(train_data.classes[label])
print(image[0, :3, :3])                  # the top-left 3x3 pixels of the red channel

Output

torch.Size([3, 224, 224])
Abyssinian
tensor([[2.1290, 2.0434, 1.4783],
        [2.1290, 2.1119, 2.0777],
        [2.1119, 2.1290, 2.1119]])

The values aren't 0 to 255 any more. The preprocessing also standardised them: from each channel it subtracted the average value across ImageNet and divided by the spread, so the numbers are centred on zero, running from about −2 for black to between +2.2 and +2.6 for full brightness, depending on the channel. This photo's top-left corner is nearly white, which is why its numbers are all near the top of that range. The model was trained on numbers in that form, so we must feed it the same. Why standardising helps training is a question for Lesson 3.

What comes out: scores and probabilities

For each photo the model outputs 37 numbers, one per breed. These raw scores are called logits. They can be any real number, positive or negative, and on their own they're hard to interpret: is a score of 4.2 good?

Python

model.eval()
with torch.no_grad():
    logits = model(image.unsqueeze(0).to(device))[0]
print(logits[:5])                        # raw scores for the first 5 breeds
print(torch.softmax(logits, dim=0)[:5])  # the same, turned into probabilities

Output

tensor([ 0.0648, -4.6784, -5.6487, -6.4490, -3.7575])
tensor([0.4257, 0.0037, 0.0014, 0.0006, 0.0093])

The softmax function turns the 37 scores into 37 probabilities. That raises the first question worth asking.

What makes a list of numbers a probability distribution?

Question 1: the model's output is called a probability distribution over breeds. What does that actually require?

A probability distribution over a set of possible outcomes (here, the 37 breeds) assigns each outcome a number P with just two rules:

\begin{array}{lll} 1.\;\; P(\text{outcome}) \ge 0 & \text{for every outcome} & \text{no negative probabilities} \\[4pt] 2.\;\; \displaystyle\sum_{\text{outcomes}} P(\text{outcome}) = 1 & \text{over all outcomes} & \text{something must happen} \end{array}

Everything else about probability follows from these two rules. For example, the probability that the photo shows one of several breeds is the sum of their probabilities, and the probability that it isn't a particular breed is 1 minus that breed's probability. A distribution over a list of separate outcomes like this is called a discrete distribution. Later lessons meet continuous distributions, where the outcome can be any number, such as a height or a pixel value.

The logits break both rules: they can be negative, and they don't sum to 1. So we need a function that turns any list of real numbers into a list that obeys both.

Why does softmax use the exponential?

Question 2: why is softmax defined the way it is?

\operatorname{softmax}(z)_i = \frac{e^{z_i}}{\sum_j e^{z_j}}

Each design choice solves one problem. Raising e to the power of a score makes it positive whatever the score's sign, which satisfies rule 1. Dividing by the total makes the results sum to exactly 1, which satisfies rule 2. And because eˣ always increases with x, the order of the scores is preserved: the breed with the highest score always gets the highest probability.

Why not simply divide each score by the sum of the scores? Because scores can be negative, so the "probabilities" could be negative, and the sum could even be zero. Why not square the scores to make them positive? Because then a score of −5 would count the same as +5. The exponential is the natural choice that is positive, order-preserving and smooth.

It has one more property that matters in practice: adding the same constant to every score changes nothing. Multiply the top and bottom by e^(−c) and it cancels:

\frac{e^{z_i - c}}{\sum_j e^{z_j - c}} = \frac{e^{-c}\, e^{z_i}}{e^{-c} \sum_j e^{z_j}} = \operatorname{softmax}(z)_i

So only the differences between scores matter. That's why a score of 4.2 means nothing on its own: what counts is how far it is above the others. Implementations use this property to avoid overflow, subtracting the largest score before exponentiating. Let's check all of it:

Python

def softmax(z):
    e = torch.exp(z - z.max())          # subtracting the max changes nothing (shown above)
    return e / e.sum()

z = torch.tensor([2.0, 1.0, -1.0, 4.2])
p_ = softmax(z)
print(p_, "sum:", p_.sum().item())                     # all positive, sum to 1
print(torch.allclose(softmax(z + 100), p_))            # shifting every score: same result
print(torch.allclose(p_, torch.softmax(z, dim=0)))     # matches PyTorch's softmax
print(torch.softmax(torch.tensor([1000.0, 999.0]), dim=0))   # no overflow, thanks to the max trick

Output

tensor([0.0958, 0.0352, 0.0048, 0.8642]) sum: 1.0
True
True
tensor([0.7311, 0.2689])

How confident should the model be?

Question 3: if only the differences between scores matter, what happens when the differences are bigger or smaller?

Dividing every score by a number T, called the temperature, before the softmax scales all the differences. A small T stretches them, so the top breed takes almost all of the probability. A large T shrinks them, so the probability spreads out towards the uniform distribution, 1/37 for every breed:

Python

for T in [0.25, 1.0, 4.0, 100.0]:
    print(f"T = {T:<6} probabilities: {softmax(z / T).numpy().round(3)}")
print("uniform would be", round(1 / 4, 3), "for each of 4 outcomes")

Output

T = 0.25   probabilities: [0. 0. 0. 1.]
T = 1.0    probabilities: [0.096 0.035 0.005 0.864]
T = 4.0    probabilities: [0.251 0.195 0.119 0.435]
T = 100.0  probabilities: [0.251 0.249 0.244 0.257]
uniform would be 0.25 for each of 4 outcomes

The temperature doesn't change which breed wins; it changes how sure the distribution sounds. You'll meet it again when language models generate text, where it controls how adventurous their word choices are.

What the model can't tell you

Question 4: what does the model say about a photo that isn't a cat or dog at all?

Try uploading a photo of a car, or a cup of tea. The model will still name a breed, sometimes with high confidence. It isn't broken. Rule 2 forces the probabilities to sum to 1 across the 37 breeds, so the model has no way to say "none of the above". It can only distribute its certainty among the options it was given. A probability from a classifier means "how likely this breed is, assuming the photo shows one of these 37 breeds", which is a conditional probability, and the assumption is invisible in the output.

This is one of the most important practical lessons in machine learning: a model's confidence is not the same as being correct, especially on inputs unlike its training data. Measuring how good a model really is gets a whole lesson of its own (Lesson 4).

Summary

With transfer learning you can train an accurate image classifier in minutes: take a network pretrained on ImageNet, freeze it, and replace its last layer with one for your own classes. A photo enters as a tensor of numbers (3 × 224 × 224 after preprocessing), and the model outputs one raw score, a logit, per class. Softmax, e^(zᵢ)/Σe^(zⱼ), turns logits into a probability distribution: non-negative numbers that sum to 1. Only differences between logits matter, temperature controls how confident the distribution sounds, and because the probabilities must sum to 1 over the known classes, a classifier will confidently label things it has never seen.

Exercises

1. Change the number of epochs to 1 and to 6. How does test accuracy change, and when does it stop improving?

2. Unfreeze the whole network (skip the loop that sets requires_grad = False and give the optimizer model.parameters() with lr=1e-4). Does accuracy improve? How much longer does an epoch take, and why?

3. Find the test photos the model gets most confidently wrong: the ones where the probability of its wrong prediction is highest. Look at them. Are the mistakes reasonable?

4. Show that softmax of two scores, [a, b], gives the probability 1 / (1 + e^(b − a)) for the first one. Check it numerically for a few values.

5. Upload five photos that are not cats or dogs. Record the top probability for each. Then compare with the top probability on five real pet photos. Could you use the top probability to detect "not a pet"? Where would you set the threshold, and how often would it fail?