Step 03 of 16Day 3 · Wed 28 OctWeek 1Phase A · notebooks

Machine learning basics

Learn the training loop on a toy dataset before touching your own data: splits, epochs, loss curves, and what overfitting looks like when you cause it on purpose.

Runs on: JupyterLab on the Jetson
Notebooks: 02_ml_basics.ipynb
Creates: data/fashion/ (downloaded), loss plots
Time: 4 hours, including course time

Where Where to implement

Notebook

~/sutraai/notebooks/02_ml_basics.ipynb
Kernel: SutraAI (venv)

Build log

Write 3 sentences in your own words: epoch, loss, overfitting

Do Do it

1Check the kernel before anything else

Make this the first cell of every notebook from now on. It finds out which Python the notebook runs and which matplotlib it would load, without importing it. If you already hit the 'RcParams' object has no attribute '_get' error in this notebook, click Kernel → Restart Kernel first: the failed import leaves Ubuntu's matplotlib cached in memory, and re-running the cell keeps reusing it.

Notebook cellnotebooks/02_ml_basics.ipynb (first cell)

import sys, importlib.util
spec = importlib.util.find_spec("matplotlib")             # locates matplotlib without importing it
mpl = spec.origin if spec else "not installed"
print("kernel python:", sys.executable)                   # /home/<you>/sutraai/.venv/bin/python
print("matplotlib   :", mpl)                              # /home/<you>/sutraai/.venv/lib/python3.10/site-packages/matplotlib/__init__.py
print("Ubuntu paths :", [p for p in sys.path if "dist-packages" in p])   # [] in an isolated venv
assert "/sutraai/.venv/" in sys.executable, "Wrong kernel: Kernel → Change Kernel → SutraAI (venv · torch 2.8 · CUDA 12.6)"
assert not [p for p in sys.path if "dist-packages" in p], "The venv still sees Ubuntu's packages: do the one-time switch in Step 02, substep 2"
assert "/sutraai/.venv/" in mpl, "matplotlib missing from the venv: pip install matplotlib==3.10.9 (venv active), then restart the kernel"
print("✅ kernel OK")

2Load a small dataset and split it three ways

Train is what the model learns from. Validation is what you tune against. Test is the final honest score, used once at the end.

Notebook cellnotebooks/02_ml_basics.ipynb

import torch, torch.nn as nn, matplotlib.pyplot as plt
from torchvision import datasets, transforms
from torch.utils.data import DataLoader, Subset, random_split

torch.manual_seed(42)
tf = transforms.ToTensor()
full = datasets.FashionMNIST("../data/fashion", train=True, download=True, transform=tf)
test = datasets.FashionMNIST("../data/fashion", train=False, download=True, transform=tf)

small = Subset(full, range(6000))                     # small on purpose: fast, and easy to overfit
train_ds, val_ds = random_split(small, [5000, 1000])
train_dl = DataLoader(train_ds, batch_size=64, shuffle=True)
val_dl = DataLoader(val_ds, batch_size=256)
test_dl = DataLoader(test, batch_size=256)
print(len(train_ds), len(val_ds), len(test))

3Write the training loop yourself

Notebook cellnotebooks/02_ml_basics.ipynb

def make_model(hidden=256, dropout=0.0):
    return nn.Sequential(nn.Flatten(), nn.Linear(28 * 28, hidden), nn.ReLU(),
                         nn.Dropout(dropout), nn.Linear(hidden, 10)).cuda()

def run_epoch(model, dl, opt=None):
    train = opt is not None
    model.train(train)
    total, correct, n = 0.0, 0, 0
    with torch.set_grad_enabled(train):
        for x, y in dl:
            x, y = x.cuda(), y.cuda()
            out = model(x)
            loss = nn.functional.cross_entropy(out, y)
            if train:
                opt.zero_grad(); loss.backward(); opt.step()
            total += loss.item() * len(x); correct += (out.argmax(1) == y).sum().item(); n += len(x)
    return total / n, correct / n

def train(model, epochs=20, lr=1e-3):
    opt = torch.optim.Adam(model.parameters(), lr=lr)
    hist = {"train": [], "val": []}
    for ep in range(epochs):
        tr_loss, _ = run_epoch(model, train_dl, opt)
        va_loss, va_acc = run_epoch(model, val_dl)
        hist["train"].append(tr_loss); hist["val"].append(va_loss)
        print(f"epoch {ep+1:2d}  train loss {tr_loss:.3f}  val loss {va_loss:.3f}  val acc {va_acc:.3f}")
    return hist

model = make_model()
hist = train(model, epochs=20)
plt.plot(hist["train"], label="train"); plt.plot(hist["val"], label="validation")
plt.xlabel("epoch"); plt.ylabel("loss"); plt.legend(); plt.show()

4Overfit on purpose

Train a much bigger network for longer. Watch the training loss keep falling while the validation loss turns upward: the model is memorising instead of learning.

Notebook cellnotebooks/02_ml_basics.ipynb

big = make_model(hidden=2048)
hist_big = train(big, epochs=60)
plt.plot(hist_big["train"], label="train"); plt.plot(hist_big["val"], label="validation")
plt.title("Overfitting: validation loss rises"); plt.legend(); plt.show()

5Fix it and get the honest score

Two common fixes: dropout (randomly switch off neurons during training) and early stopping (keep the epoch with the best validation score). Then measure on the test set once.

Notebook cellnotebooks/02_ml_basics.ipynb

fixed = make_model(hidden=512, dropout=0.3)
best, best_state = 0.0, None
opt = torch.optim.Adam(fixed.parameters(), lr=1e-3)
for ep in range(30):
    run_epoch(fixed, train_dl, opt)
    _, va_acc = run_epoch(fixed, val_dl)
    if va_acc > best:
        best, best_state = va_acc, {k: v.clone() for k, v in fixed.state_dict().items()}
fixed.load_state_dict(best_state)
test_loss, test_acc = run_epoch(fixed, test_dl)
print(f"best val acc {best:.3f} | test acc {test_acc:.3f}")

Test Test it

Notebook cellnotebooks/02_ml_basics.ipynb (last cell)

assert test_acc > 0.75, f"test accuracy too low: {test_acc:.3f}"
assert min(hist_big["val"]) < hist_big["val"][-1], "expected the big model to overfit (val loss rises at the end)"
print("✅ basics understood")

Pass: the last cell prints "✅ basics understood" after Restart and Run All, and you can point at the epoch where overfitting starts on the plot.

Done Done when

Kernel check prints "✅ kernel OK" (venv Python, venv matplotlib)
You can explain train, validation and test splits in one sentence each
Loss curve plotted for the small model
Overfitting reproduced and visible on the plot
Dropout + early stopping version scores above 75% on test
BUILDLOG entry written in your own words

If it breaks

Symptom	Fix
`AttributeError: 'RcParams' object has no attribute '_get'` on the first import line	The notebook loaded Ubuntu's old matplotlib (path `/usr/lib/python3/dist-packages`) because the venv can see Ubuntu's packages. Do the one-time switch to an isolated venv in Step 02, substep 2, restart JupyterLab, then Kernel → Restart Kernel and run the substep 1 check. Re-running a cell without a restart always repeats the error.
Check says "Wrong kernel"	Top-right kernel name must read SutraAI (venv · torch 2.8 · CUDA 12.6). If it isn't in the list, in the Jetson terminal: `source ~/sutraai/.venv/bin/activate && python -m ipykernel install --user --name sutraai --display-name "SutraAI (venv · torch 2.8 · CUDA 12.6)"`, then reload the browser tab.
Download fails	Check the internet on the Jetson; or download FashionMNIST on the laptop and copy `data/fashion` over with `scp -r`.
Big model doesn't overfit	Use fewer samples (`range(2000)`) or more epochs.
Kernel restarts	Close other notebooks; check RAM in jtop.

Learn more

ZTM "PyTorch for Deep Learning": fundamentals and workflow sections (course page, also on Udemy)
learnpytorch.io: chapter 01, PyTorch workflow (free)