Free course · by the SutraCore team

Free NVIDIA Jetson AI course: 30 days, 16 steps

Orin NX 16GBJetPack 6.2.1CUDA 12.6TensorRT 10.3Python 3.10PyTorch 2.8
Step 04 of 16Days 4–5 · Thu 29 – Fri 30 OctWeek 1Phase A · notebooks

Your first image model

Photograph fabric with and without defects (or small parts), split the photos cleanly, and fine-tune a pre-trained MobileNet. This is template T5 (Product Defect QC), and the same code serves T3 (Chest X-ray) later.

Runs on
Phone camera → laptop → JupyterLab on the Jetson
Day 4
Morning: prepare + shoot 180 photos. Afternoon: NB10 + first NB11 run. Day 5: fix the data, rerun.
Notebooks
10_image_dataset.ipynb, 11_image_train.ipynb
Creates
data/processed/images/, models/image_v1/best.pt, reports/image_v1/metrics.json
Time
Two days

Where Where to implement

Data

  • Laptop copy: C:\sutraai-photos\{clean,stain,hole}\
  • Raw photos on the Jetson: ~/sutraai/data/raw/images/<class>/*.jpg
  • Splits (512 px copies): data/processed/images/{train,val,test}/<class>/

Code

  • Notebooks NB10 and NB11
  • On Day 5, move the reusable parts into sutra_core/data.py and sutra_core/train.py

Do Do it

1Choose what to photograph (5 minutes)

Photograph small objects on a table, not people and not furniture. This step is template T5, Product Defect QC: the model looks at one item in the photo and says which class it belongs to. Pick one option and stick to it for the whole step.

OptionClasses = folder namesWhat you needWhy
A · Fabric QC (recommended)clean · stain · hole2–3 old cotton cloths (old T-shirt, towel, dupatta or a piece of an old bedsheet), tea / turmeric / ink for stains, scissorsA real factory job (textile inspection), and you make the defects yourself, so you control every class
B · Part sortingscrew · nut · washer
or resistor · capacitor · led
15+ pieces of each from a toolbox or your electronics drawerEasiest way to reach high accuracy; nothing to damage
Don't use nowWhy
People and facesPersonal data that needs consent. Faces come in Step 09 (driver drowsiness), with their own rules.
A whole bed, sofa or roomToo big: at 96×96 pixels the model can't see a stain on a whole bed. A bedsheet is fine as fabric, but photograph a 20×20 cm patch, not the bed.
Photos from the internetThe point is learning to collect your own data, the way SutraAI's users will.

2Prepare the items (Option A: 20 minutes)

  1. Cut the cloth into about 15 pieces of 20×20 cm. Mix plain and printed, light and dark.
  2. Use the same pieces for all three classes, in this order: photograph a piece while it's clean → put a stain on one area → photograph it → cut or tear a hole in a different area → photograph that. Then every class has the same colours and patterns, and the only difference the model can learn is the defect.
  3. Make at least 12 different stains (tea, coffee, turmeric, ink, oil, mud) and 12 different holes (cut and torn, round and slit). Each one at least 1.5 cm across, about the size of your fingertip, or it disappears when the photo is shrunk to 96×96.

Option B: put 1–3 pieces of the same class in each photo, never mixed classes, and use different individual pieces across photos.

3Shoot 60 photos per class (45 minutes, phone camera)

Shoot 15 photos per class in each of the four blocks below: 60 per class, 180 in total. In every block, shoot all three classes before moving on.

BlockLightBackgroundPer class
1Daylight near a windowTable top15
2Room LED or tube light (evening)Table top15
3Daylight near a windowA different surface: floor tiles, newspaper or a bedsheet15
4Room LED or tube lightThat different surface15

Within each block of 15, change something in every photo: distance (close, about 15 cm, or medium, about 30 cm), angle (straight down or tilted 30–45°), rotate the item, move the defect to another spot. Keep the item and the defect in the middle two-thirds of the frame, because training crops the edges.

Golden rule: every light and every background must appear in every class. If all the stains are shot on the table and all the holes on the floor, the model learns "floor = hole", scores 100%, and fails on real cloth.

A clean photo must have no stain or hole anywhere in it. One class per photo.

Phone camera settings
ModeNormal Photo mode, 1× lens. No portrait, beauty, night mode or filters. Flash off.
FocusTap on the defect (or the part) before each shot. Delete blurry photos straight away.
No burstsDon't take near-identical photos. Move or change something between shots.
File typeJPG. On an iPhone: Settings → Camera → Formats → Most Compatible (otherwise you get HEIC files the notebook skips).
TransferUSB cable, or Google Photos / Drive with "download original". Not WhatsApp: it compresses the photos.

4Sort into class folders and copy to the Jetson

On the laptop, make one folder per class. The folder names become the class names, so use lowercase with no spaces.

FileC:\sutraai-photos (on the laptop)
C:\sutraai-photos\
├── clean\     60 photos (.jpg)
├── stain\     60 photos
└── hole\      60 photos
Laptop terminalWindows PowerShell
ssh <your-user>@<jetson-ip> "mkdir -p ~/sutraai/data/raw/images"
scp -r C:\sutraai-photos\clean C:\sutraai-photos\stain C:\sutraai-photos\hole <your-user>@<jetson-ip>:~/sutraai/data/raw/images/
Jetson terminal
cd ~/sutraai/data/raw/images
for d in */; do printf "%-10s %s photos\n" "$d" "$(find "$d" -maxdepth 1 -type f \( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \) | wc -l)"; done
du -sh .        # 180 phone photos ≈ 0.5–1 GB
# expect: clean/ 60 photos, stain/ 60 photos, hole/ 60 photos (at least 50 each, roughly equal)

Only your class folders may live in data/raw/images. Left-over folders from a test run (for example good/ and defect/) would otherwise turn into extra classes. NB10 now uses only the folders named in CLASSES and lists the rest as ignored, but park them elsewhere to keep things clear:

Jetson terminal
mkdir -p ~/sutraai/data/raw/_parked
mv -n ~/sutraai/data/raw/images/good ~/sutraai/data/raw/images/defect ~/sutraai/data/raw/_parked/
ls ~/sutraai/data/raw/images          # expect: clean  hole  stain

How many photos do you have?

Photos per classWhat you can do
Under 10NB10 stops. Shoot more.
10–49 (e.g. one block of 15 done)Dry run: NB10 and NB11 run end to end so you learn the flow, but the score means little. With 17 photos per class the test set holds only 4 per class, so one wrong photo moves accuracy by about 8 points. The ≥85% test target doesn't apply yet.
50–60+ (all 4 blocks done)Real model: the test cell's targets apply.

To add photos later, shoot the missing blocks, copy them into the same class folders with the same scp command, and re-run NB10. It rebuilds the splits from scratch every time.

5NB10: shrink, split 70 / 15 / 15 and check for leaks

Phone photos are 12 megapixels. Decoding them every epoch is slow on the Jetson's CPU, so NB10 saves 512-pixel copies (plenty for a 96-pixel model) and fixes the phone's rotation flag. Your originals in data/raw stay untouched.

Notebook cellnotebooks/10_image_dataset.ipynb
import random, shutil, hashlib
from pathlib import Path
import matplotlib.pyplot as plt
from PIL import Image, ImageOps

ROOT = Path.home() / "sutraai"
RAW, OUT = ROOT / "data/raw/images", ROOT / "data/processed/images"
CLASSES = ["clean", "stain", "hole"]             # Option B: ["screw", "nut", "washer"]. Only these folders are used.
EXTS = {".jpg", ".jpeg", ".png"}                 # compared in lower case, so .JPG from iPhones counts too
MAX_SIDE = 512
random.seed(42)
if OUT.exists(): shutil.rmtree(OUT)

def save_small(src, dst):
    img = ImageOps.exif_transpose(Image.open(src)).convert("RGB")   # apply the phone's rotation flag
    img.thumbnail((MAX_SIDE, MAX_SIDE))                              # longest side 512 px, keeps the shape
    img.save(dst.with_suffix(".jpg"), quality=92)

found = sorted(p.name for p in RAW.iterdir() if p.is_dir())
ignored, missing = [c for c in found if c not in CLASSES], [c for c in CLASSES if c not in found]
if ignored: print(f"⚠ ignoring folders not in CLASSES: {ignored} (park them outside data/raw/images, substep 4)")
assert not missing, f"no folder for {missing} in {RAW}"

counts, first_seen, conflicts = {}, {}, []
for cls_dir in [RAW / c for c in CLASSES]:
    files = sorted(f for f in cls_dir.iterdir() if f.is_file() and f.suffix.lower() in EXTS)
    skipped = [f.name for f in cls_dir.iterdir() if f.is_file() and f.suffix.lower() not in EXTS]
    if skipped: print(f"{cls_dir.name}: skipped {len(skipped)} files, e.g. {skipped[0]} (convert HEIC to JPG)")
    unique = []                                       # the same photo copied twice would leak across splits
    for f in files:
        h = hashlib.sha256(f.read_bytes()).hexdigest()
        if h in first_seen:
            other = first_seen[h]
            if other.parent == f.parent: print(f"  {cls_dir.name}: duplicate skipped, {f.name} = {other.name}")
            else: conflicts.append(f"{other.parent.name}/{other.name} = {f.parent.name}/{f.name}")
            continue
        first_seen[h] = f; unique.append(f)
    files = unique
    random.shuffle(files)
    n = len(files); n_tr, n_va = int(n * 0.70), int(n * 0.15)
    assert n >= 10, f"'{cls_dir.name}': only {n} JPG/PNG photos found in {cls_dir}. Need 50+ (aim for 60)."
    if n < 50: print(f"⚠ '{cls_dir.name}': {n} photos is enough for a dry run, not for a real model (take 50+)")
    for split, chunk in [("train", files[:n_tr]), ("val", files[n_tr:n_tr + n_va]), ("test", files[n_tr + n_va:])]:
        dst = OUT / split / cls_dir.name; dst.mkdir(parents=True, exist_ok=True)
        for f in chunk: save_small(f, dst / f.name)
    counts[cls_dir.name] = n
    print(f"{cls_dir.name:10} {n:4d} photos → train {n_tr}, val {n_va}, test {n - n_tr - n_va}")
assert not conflicts, "the same photo is in two class folders, delete the wrong copy: " + "; ".join(conflicts[:5])
print(counts)                                                      # e.g. {'clean': 60, 'hole': 60, 'stain': 60}
plt.bar(counts.keys(), counts.values()); plt.title("images per class"); plt.show()
Notebook cellnotebooks/10_image_dataset.ipynb
# the same photo must never be in two splits
seen = {}
for p in OUT.rglob("*.jpg"):
    seen.setdefault(hashlib.sha256(p.read_bytes()).hexdigest(), []).append(p.relative_to(OUT))
leaks = [paths for paths in seen.values() if len({q.parts[0] for q in paths}) > 1]
print("duplicate photos across splits:", len(leaks))                # expect: 0
for paths in leaks[:5]: print("   ", " = ".join(map(str, paths)))   # names, if any, so you can delete one copy

# look at a few samples per class
fig, axes = plt.subplots(len(counts), 5, figsize=(12, 2.6 * len(counts)), squeeze=False)
for r, cls in enumerate(counts):
    for c, f in enumerate(sorted((OUT / "train" / cls).iterdir())[:5]):
        axes[r][c].imshow(Image.open(f)); axes[r][c].set_title(cls); axes[r][c].axis("off")
plt.show()

6NB11: fine-tune MobileNetV2 (transfer learning)

The model already knows edges, textures and shapes from millions of ImageNet photos. You only teach it your classes. 96×96 input keeps it small enough for microcontrollers.

Notebook cellnotebooks/11_image_train.ipynb
import json, timm, torch, numpy as np
from pathlib import Path
from torchvision import datasets, transforms as T
from torch.utils.data import DataLoader
from sklearn.metrics import confusion_matrix, classification_report

ROOT = Path.home() / "sutraai"
CFG = dict(size=96, arch="mobilenetv2_050", epochs=30, lr=1e-3, batch=32, seed=42, run="image_v1")
torch.manual_seed(CFG["seed"]); np.random.seed(CFG["seed"])
(ROOT / "models" / CFG["run"]).mkdir(parents=True, exist_ok=True)
(ROOT / "reports" / CFG["run"]).mkdir(parents=True, exist_ok=True)

mean, std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]
train_tf = T.Compose([T.RandomResizedCrop(CFG["size"], scale=(0.7, 1.0)), T.RandomHorizontalFlip(),
                      T.ColorJitter(0.3, 0.3, 0.2), T.ToTensor(), T.Normalize(mean, std)])
eval_tf = T.Compose([T.Resize(int(CFG["size"] * 1.15)), T.CenterCrop(CFG["size"]), T.ToTensor(), T.Normalize(mean, std)])
D = ROOT / "data/processed/images"
for split in ["train", "val", "test"]:                         # friendly check before ImageFolder complains
    empty = [c.name for c in sorted((D / split).iterdir()) if not any(c.iterdir())]
    assert not empty, f"{split}/ has no photos for {empty}: too few photos per class. Add photos, re-run NB10."
train_ds = datasets.ImageFolder(D / "train", train_tf)
val_ds, test_ds = datasets.ImageFolder(D / "val", eval_tf), datasets.ImageFolder(D / "test", eval_tf)
classes = train_ds.classes
loader = lambda ds, shuffle: DataLoader(ds, batch_size=CFG["batch"], shuffle=shuffle, num_workers=4)

model = timm.create_model(CFG["arch"], pretrained=True, num_classes=len(classes)).cuda()
opt = torch.optim.AdamW(model.parameters(), lr=CFG["lr"])
sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, CFG["epochs"])
scaler = torch.amp.GradScaler()

def predict(ds):
    model.eval(); preds, labels = [], []
    with torch.no_grad():
        for x, y in loader(ds, False):
            preds += model(x.cuda()).argmax(1).cpu().tolist(); labels += y.tolist()
    return np.array(labels), np.array(preds)
Notebook cellnotebooks/11_image_train.ipynb
best = 0.0
for epoch in range(CFG["epochs"]):
    model.train()
    for x, y in loader(train_ds, True):
        x, y = x.cuda(), y.cuda()
        with torch.autocast("cuda", dtype=torch.float16):          # mixed precision: faster, less memory
            loss = torch.nn.functional.cross_entropy(model(x), y)
        opt.zero_grad(); scaler.scale(loss).backward(); scaler.step(opt); scaler.update()
    sched.step()
    yv, pv = predict(val_ds); val_acc = float((yv == pv).mean())
    print(json.dumps({"epoch": epoch + 1, "loss": round(loss.item(), 4), "val_acc": round(val_acc, 4)}))  # one JSON line per epoch: the web UI will read these later
    if val_acc > best:
        best = val_acc
        torch.save(model.state_dict(), ROOT / "models" / CFG["run"] / "best.pt")

7Score on the test set and save metrics.json

Notebook cellnotebooks/11_image_train.ipynb
model.load_state_dict(torch.load(ROOT / "models" / CFG["run"] / "best.pt"))
yt, pt = predict(test_ds)
metrics = {
    "run": CFG["run"], "config": CFG, "classes": classes,
    "val_acc": best, "test_acc": float((yt == pt).mean()),
    "confusion_matrix": confusion_matrix(yt, pt).tolist(),
    "per_class": classification_report(yt, pt, target_names=classes, output_dict=True),
}
json.dump(metrics, open(ROOT / "reports" / CFG["run"] / "metrics.json", "w"), indent=2)
print("test accuracy:", round(metrics["test_acc"], 3))
print(np.array(metrics["confusion_matrix"]))

8Look at the mistakes (Day 5)

Most errors are data problems: a blurry photo, a wrong folder, a class that looks too similar. Fix the data, not the code.

Notebook cellnotebooks/11_image_train.ipynb
import matplotlib.pyplot as plt
from PIL import Image
wrong = [i for i, (a, b) in enumerate(zip(yt, pt)) if a != b]
fig, axes = plt.subplots(1, max(1, min(6, len(wrong))), figsize=(14, 3), squeeze=False)
for ax, i in zip(axes[0], wrong[:6]):
    path, _ = test_ds.samples[i]
    ax.imshow(Image.open(path)); ax.set_title(f"true {classes[yt[i]]}\npred {classes[pt[i]]}"); ax.axis("off")
plt.show()

Then move predict, the transforms and the training loop into sutra_core/train.py, and import them with %load_ext autoreload + %autoreload 2 at the top of the notebook.

Test Test it

These are two separate cells in two notebooks. Each notebook only knows its own variables: leaks and counts live in NB10, metrics lives in NB11.

Notebook cellnotebooks/10_image_dataset.ipynb (last cell)
assert not leaks, f"{len(leaks)} photos appear in more than one split: {leaks[:3]}"
assert all(len(list((OUT / "train" / c).iterdir())) >= 35 for c in counts), f"need at least 50 photos per class, have {counts} (a dry run stops here: expected)"
assert max(counts.values()) <= 1.5 * min(counts.values()), f"classes unbalanced {counts}: shoot more of the smallest class"
print("✅ dataset OK")
Notebook cellnotebooks/11_image_train.ipynb (last cell)
assert metrics["test_acc"] >= 0.85, f"test accuracy {metrics['test_acc']:.3f}: add more varied photos or check labels"
print("✅ image model OK")

Reproducibility check: Restart and Run All in NB11 a second time. The test accuracy should land within about 1 point of the first run.

Pass: no leaks, test accuracy ≥ 85% on your photos, a metrics.json with a confusion matrix, and the second run agrees.

Done Done when

  • One option chosen (A: clean / stain / hole, or B: three part types)
  • About 60 photos per class across all 4 light × background blocks, copied to the Jetson
  • Clean 70/15/15 split with zero duplicates across splits
  • best.pt and metrics.json saved
  • Test accuracy ≥ 85%
  • Misclassified photos reviewed and data fixed where needed
  • Helpers moved into sutra_core/ and imported by the notebook

If it breaks

SymptomFix
UserWarning: Unable to import Axes3D when NB10 imports matplotlibThe venv can still see Ubuntu's old matplotlib. Do the one-time switch to an isolated venv in Step 02, substep 2, restart JupyterLab and the kernel.
FileNotFoundError: Found no valid file for the classes … on the val or test line of NB11Too few photos per class reached NB10, so the 15% val/test share rounded down to zero. Usually: only a handful of photos copied, or the old NB10 skipped .JPG/.HEIC files. Count what's really there with the cell below this table, add photos (50+ per class), re-run the current NB10, then NB11.
Warning: You are sending unauthenticated requests to the HF Hub in NB11Harmless: timm downloads the pretrained MobileNet from Hugging Face without logging in, and it still works (cached after the first run). To remove the warning and get higher download limits, log in once with a free Read token: hf auth login (Step 02, substep 5), then restart the kernel.
NameError: name 'leaks' is not defined (or 'metrics')The NB10 checks were pasted into NB11, or the other way round. The Test section has one cell for the end of NB10 and a separate cell for the end of NB11.
AssertionError: … photos appear in more than one splitThe same picture exists twice, e.g. copied from the phone twice (IMG_1234.jpg and IMG_1234 (1).jpg). The current NB10 skips exact duplicates inside a class and stops if one photo sits in two class folders, naming both files. Copy the current NB10, re-run it, then the leak cell prints any remaining pairs.
NB10 shows extra classes, e.g. {'clean': 17, 'defect': 3, 'good': 5, 'hole': 16, 'stain': 17}Old test folders are still in data/raw/images. Park them in data/raw/_parked (substep 4) and check CLASSES at the top of NB10. Then re-run NB10.
NB10 says "skipped … .heic"iPhone HEIC files. Set Camera → Formats → Most Compatible and reshoot, or convert them to JPG on the laptop before copying.
100% accuracy on the first trySuspicious. Check that every class was shot in all 4 blocks; the model may be reading the background or light instead of the defect. Test it on a new cloth photographed in a new spot.
stain and hole get mixed upDefects too small or too far away. Shoot closer (15 cm) and make defects at least 1.5 cm across. View a photo at thumbnail size: if you can't see the defect, neither can the model.
Photos look sideways in the sample gridRun the new NB10 again: ImageOps.exif_transpose applies the phone's rotation.
Accuracy stuck near chanceCheck folder names and labels; open a few images per class; lower the learning rate to 3e-4.
Train accuracy high, test lowOverfitting: more varied photos, stronger augmentation, fewer epochs.
"Killed" or kernel diesLower batch to 16, set num_workers=0, close other kernels.
Pretrained weights won't downloadDownload once on the laptop (timm cache) and copy it, or check the Jetson's internet.
Grayscale X-ray images later (T3)Add T.Grayscale(num_output_channels=3) first in both transforms.

Count what each folder really holds, and which file types are in your raw folders:

Notebook cellany notebook (diagnostic)
from pathlib import Path
from collections import Counter
ROOT = Path.home() / "sutraai"
for c in sorted(p for p in (ROOT / "data/raw/images").iterdir() if p.is_dir()):
    kinds = Counter(f.suffix for f in c.iterdir() if f.is_file())
    print(f"raw   {c.name:10} {sum(kinds.values()):4d} files  {dict(kinds)}")     # e.g. {'.jpg': 58, '.JPG': 2}
for split in ["train", "val", "test"]:
    d = ROOT / "data/processed/images" / split
    if d.exists():
        print(split.ljust(5), {c.name: len(list(c.iterdir())) for c in sorted(d.iterdir())})

Learn more

  • ZTM "PyTorch for Deep Learning": computer vision, custom datasets and transfer learning sections (course page)
  • learnpytorch.io: chapters 03–06 (free)
  • Free: NVIDIA DLI "Getting Started With AI on Jetson Nano" (image classification on Jetson), on NVIDIA AI Learning Essentials