Your first image model
Photograph fabric with and without defects (or small parts), split the photos cleanly, and fine-tune a pre-trained MobileNet. This is template T5 (Product Defect QC), and the same code serves T3 (Chest X-ray) later.
Where Where to implement
Data
- Laptop copy:
C:\sutraai-photos\{clean,stain,hole}\ - Raw photos on the Jetson:
~/sutraai/data/raw/images/<class>/*.jpg - Splits (512 px copies):
data/processed/images/{train,val,test}/<class>/
Code
- Notebooks NB10 and NB11
- On Day 5, move the reusable parts into
sutra_core/data.pyandsutra_core/train.py
Do Do it
1Choose what to photograph (5 minutes)
Photograph small objects on a table, not people and not furniture. This step is template T5, Product Defect QC: the model looks at one item in the photo and says which class it belongs to. Pick one option and stick to it for the whole step.
| Option | Classes = folder names | What you need | Why |
|---|---|---|---|
| A · Fabric QC (recommended) | clean · stain · hole | 2–3 old cotton cloths (old T-shirt, towel, dupatta or a piece of an old bedsheet), tea / turmeric / ink for stains, scissors | A real factory job (textile inspection), and you make the defects yourself, so you control every class |
| B · Part sorting | screw · nut · washeror resistor · capacitor · led | 15+ pieces of each from a toolbox or your electronics drawer | Easiest way to reach high accuracy; nothing to damage |
| Don't use now | Why |
|---|---|
| People and faces | Personal data that needs consent. Faces come in Step 09 (driver drowsiness), with their own rules. |
| A whole bed, sofa or room | Too big: at 96×96 pixels the model can't see a stain on a whole bed. A bedsheet is fine as fabric, but photograph a 20×20 cm patch, not the bed. |
| Photos from the internet | The point is learning to collect your own data, the way SutraAI's users will. |
2Prepare the items (Option A: 20 minutes)
- Cut the cloth into about 15 pieces of 20×20 cm. Mix plain and printed, light and dark.
- Use the same pieces for all three classes, in this order: photograph a piece while it's clean → put a stain on one area → photograph it → cut or tear a hole in a different area → photograph that. Then every class has the same colours and patterns, and the only difference the model can learn is the defect.
- Make at least 12 different stains (tea, coffee, turmeric, ink, oil, mud) and 12 different holes (cut and torn, round and slit). Each one at least 1.5 cm across, about the size of your fingertip, or it disappears when the photo is shrunk to 96×96.
Option B: put 1–3 pieces of the same class in each photo, never mixed classes, and use different individual pieces across photos.
3Shoot 60 photos per class (45 minutes, phone camera)
Shoot 15 photos per class in each of the four blocks below: 60 per class, 180 in total. In every block, shoot all three classes before moving on.
| Block | Light | Background | Per class |
|---|---|---|---|
| 1 | Daylight near a window | Table top | 15 |
| 2 | Room LED or tube light (evening) | Table top | 15 |
| 3 | Daylight near a window | A different surface: floor tiles, newspaper or a bedsheet | 15 |
| 4 | Room LED or tube light | That different surface | 15 |
Within each block of 15, change something in every photo: distance (close, about 15 cm, or medium, about 30 cm), angle (straight down or tilted 30–45°), rotate the item, move the defect to another spot. Keep the item and the defect in the middle two-thirds of the frame, because training crops the edges.
Golden rule: every light and every background must appear in every class. If all the stains are shot on the table and all the holes on the floor, the model learns "floor = hole", scores 100%, and fails on real cloth.
A clean photo must have no stain or hole anywhere in it. One class per photo.
| Phone camera settings | |
|---|---|
| Mode | Normal Photo mode, 1× lens. No portrait, beauty, night mode or filters. Flash off. |
| Focus | Tap on the defect (or the part) before each shot. Delete blurry photos straight away. |
| No bursts | Don't take near-identical photos. Move or change something between shots. |
| File type | JPG. On an iPhone: Settings → Camera → Formats → Most Compatible (otherwise you get HEIC files the notebook skips). |
| Transfer | USB cable, or Google Photos / Drive with "download original". Not WhatsApp: it compresses the photos. |
4Sort into class folders and copy to the Jetson
On the laptop, make one folder per class. The folder names become the class names, so use lowercase with no spaces.
C:\sutraai-photos\
├── clean\ 60 photos (.jpg)
├── stain\ 60 photos
└── hole\ 60 photosssh <your-user>@<jetson-ip> "mkdir -p ~/sutraai/data/raw/images"
scp -r C:\sutraai-photos\clean C:\sutraai-photos\stain C:\sutraai-photos\hole <your-user>@<jetson-ip>:~/sutraai/data/raw/images/cd ~/sutraai/data/raw/images
for d in */; do printf "%-10s %s photos\n" "$d" "$(find "$d" -maxdepth 1 -type f \( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \) | wc -l)"; done
du -sh . # 180 phone photos ≈ 0.5–1 GB
# expect: clean/ 60 photos, stain/ 60 photos, hole/ 60 photos (at least 50 each, roughly equal)Only your class folders may live in data/raw/images. Left-over folders from a test run (for example good/ and defect/) would otherwise turn into extra classes. NB10 now uses only the folders named in CLASSES and lists the rest as ignored, but park them elsewhere to keep things clear:
mkdir -p ~/sutraai/data/raw/_parked
mv -n ~/sutraai/data/raw/images/good ~/sutraai/data/raw/images/defect ~/sutraai/data/raw/_parked/
ls ~/sutraai/data/raw/images # expect: clean hole stainHow many photos do you have?
| Photos per class | What you can do |
|---|---|
| Under 10 | NB10 stops. Shoot more. |
| 10–49 (e.g. one block of 15 done) | Dry run: NB10 and NB11 run end to end so you learn the flow, but the score means little. With 17 photos per class the test set holds only 4 per class, so one wrong photo moves accuracy by about 8 points. The ≥85% test target doesn't apply yet. |
| 50–60+ (all 4 blocks done) | Real model: the test cell's targets apply. |
To add photos later, shoot the missing blocks, copy them into the same class folders with the same scp command, and re-run NB10. It rebuilds the splits from scratch every time.
5NB10: shrink, split 70 / 15 / 15 and check for leaks
Phone photos are 12 megapixels. Decoding them every epoch is slow on the Jetson's CPU, so NB10 saves 512-pixel copies (plenty for a 96-pixel model) and fixes the phone's rotation flag. Your originals in data/raw stay untouched.
import random, shutil, hashlib
from pathlib import Path
import matplotlib.pyplot as plt
from PIL import Image, ImageOps
ROOT = Path.home() / "sutraai"
RAW, OUT = ROOT / "data/raw/images", ROOT / "data/processed/images"
CLASSES = ["clean", "stain", "hole"] # Option B: ["screw", "nut", "washer"]. Only these folders are used.
EXTS = {".jpg", ".jpeg", ".png"} # compared in lower case, so .JPG from iPhones counts too
MAX_SIDE = 512
random.seed(42)
if OUT.exists(): shutil.rmtree(OUT)
def save_small(src, dst):
img = ImageOps.exif_transpose(Image.open(src)).convert("RGB") # apply the phone's rotation flag
img.thumbnail((MAX_SIDE, MAX_SIDE)) # longest side 512 px, keeps the shape
img.save(dst.with_suffix(".jpg"), quality=92)
found = sorted(p.name for p in RAW.iterdir() if p.is_dir())
ignored, missing = [c for c in found if c not in CLASSES], [c for c in CLASSES if c not in found]
if ignored: print(f"⚠ ignoring folders not in CLASSES: {ignored} (park them outside data/raw/images, substep 4)")
assert not missing, f"no folder for {missing} in {RAW}"
counts, first_seen, conflicts = {}, {}, []
for cls_dir in [RAW / c for c in CLASSES]:
files = sorted(f for f in cls_dir.iterdir() if f.is_file() and f.suffix.lower() in EXTS)
skipped = [f.name for f in cls_dir.iterdir() if f.is_file() and f.suffix.lower() not in EXTS]
if skipped: print(f"{cls_dir.name}: skipped {len(skipped)} files, e.g. {skipped[0]} (convert HEIC to JPG)")
unique = [] # the same photo copied twice would leak across splits
for f in files:
h = hashlib.sha256(f.read_bytes()).hexdigest()
if h in first_seen:
other = first_seen[h]
if other.parent == f.parent: print(f" {cls_dir.name}: duplicate skipped, {f.name} = {other.name}")
else: conflicts.append(f"{other.parent.name}/{other.name} = {f.parent.name}/{f.name}")
continue
first_seen[h] = f; unique.append(f)
files = unique
random.shuffle(files)
n = len(files); n_tr, n_va = int(n * 0.70), int(n * 0.15)
assert n >= 10, f"'{cls_dir.name}': only {n} JPG/PNG photos found in {cls_dir}. Need 50+ (aim for 60)."
if n < 50: print(f"⚠ '{cls_dir.name}': {n} photos is enough for a dry run, not for a real model (take 50+)")
for split, chunk in [("train", files[:n_tr]), ("val", files[n_tr:n_tr + n_va]), ("test", files[n_tr + n_va:])]:
dst = OUT / split / cls_dir.name; dst.mkdir(parents=True, exist_ok=True)
for f in chunk: save_small(f, dst / f.name)
counts[cls_dir.name] = n
print(f"{cls_dir.name:10} {n:4d} photos → train {n_tr}, val {n_va}, test {n - n_tr - n_va}")
assert not conflicts, "the same photo is in two class folders, delete the wrong copy: " + "; ".join(conflicts[:5])
print(counts) # e.g. {'clean': 60, 'hole': 60, 'stain': 60}
plt.bar(counts.keys(), counts.values()); plt.title("images per class"); plt.show()# the same photo must never be in two splits
seen = {}
for p in OUT.rglob("*.jpg"):
seen.setdefault(hashlib.sha256(p.read_bytes()).hexdigest(), []).append(p.relative_to(OUT))
leaks = [paths for paths in seen.values() if len({q.parts[0] for q in paths}) > 1]
print("duplicate photos across splits:", len(leaks)) # expect: 0
for paths in leaks[:5]: print(" ", " = ".join(map(str, paths))) # names, if any, so you can delete one copy
# look at a few samples per class
fig, axes = plt.subplots(len(counts), 5, figsize=(12, 2.6 * len(counts)), squeeze=False)
for r, cls in enumerate(counts):
for c, f in enumerate(sorted((OUT / "train" / cls).iterdir())[:5]):
axes[r][c].imshow(Image.open(f)); axes[r][c].set_title(cls); axes[r][c].axis("off")
plt.show()6NB11: fine-tune MobileNetV2 (transfer learning)
The model already knows edges, textures and shapes from millions of ImageNet photos. You only teach it your classes. 96×96 input keeps it small enough for microcontrollers.
import json, timm, torch, numpy as np
from pathlib import Path
from torchvision import datasets, transforms as T
from torch.utils.data import DataLoader
from sklearn.metrics import confusion_matrix, classification_report
ROOT = Path.home() / "sutraai"
CFG = dict(size=96, arch="mobilenetv2_050", epochs=30, lr=1e-3, batch=32, seed=42, run="image_v1")
torch.manual_seed(CFG["seed"]); np.random.seed(CFG["seed"])
(ROOT / "models" / CFG["run"]).mkdir(parents=True, exist_ok=True)
(ROOT / "reports" / CFG["run"]).mkdir(parents=True, exist_ok=True)
mean, std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]
train_tf = T.Compose([T.RandomResizedCrop(CFG["size"], scale=(0.7, 1.0)), T.RandomHorizontalFlip(),
T.ColorJitter(0.3, 0.3, 0.2), T.ToTensor(), T.Normalize(mean, std)])
eval_tf = T.Compose([T.Resize(int(CFG["size"] * 1.15)), T.CenterCrop(CFG["size"]), T.ToTensor(), T.Normalize(mean, std)])
D = ROOT / "data/processed/images"
for split in ["train", "val", "test"]: # friendly check before ImageFolder complains
empty = [c.name for c in sorted((D / split).iterdir()) if not any(c.iterdir())]
assert not empty, f"{split}/ has no photos for {empty}: too few photos per class. Add photos, re-run NB10."
train_ds = datasets.ImageFolder(D / "train", train_tf)
val_ds, test_ds = datasets.ImageFolder(D / "val", eval_tf), datasets.ImageFolder(D / "test", eval_tf)
classes = train_ds.classes
loader = lambda ds, shuffle: DataLoader(ds, batch_size=CFG["batch"], shuffle=shuffle, num_workers=4)
model = timm.create_model(CFG["arch"], pretrained=True, num_classes=len(classes)).cuda()
opt = torch.optim.AdamW(model.parameters(), lr=CFG["lr"])
sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, CFG["epochs"])
scaler = torch.amp.GradScaler()
def predict(ds):
model.eval(); preds, labels = [], []
with torch.no_grad():
for x, y in loader(ds, False):
preds += model(x.cuda()).argmax(1).cpu().tolist(); labels += y.tolist()
return np.array(labels), np.array(preds)best = 0.0
for epoch in range(CFG["epochs"]):
model.train()
for x, y in loader(train_ds, True):
x, y = x.cuda(), y.cuda()
with torch.autocast("cuda", dtype=torch.float16): # mixed precision: faster, less memory
loss = torch.nn.functional.cross_entropy(model(x), y)
opt.zero_grad(); scaler.scale(loss).backward(); scaler.step(opt); scaler.update()
sched.step()
yv, pv = predict(val_ds); val_acc = float((yv == pv).mean())
print(json.dumps({"epoch": epoch + 1, "loss": round(loss.item(), 4), "val_acc": round(val_acc, 4)})) # one JSON line per epoch: the web UI will read these later
if val_acc > best:
best = val_acc
torch.save(model.state_dict(), ROOT / "models" / CFG["run"] / "best.pt")7Score on the test set and save metrics.json
model.load_state_dict(torch.load(ROOT / "models" / CFG["run"] / "best.pt"))
yt, pt = predict(test_ds)
metrics = {
"run": CFG["run"], "config": CFG, "classes": classes,
"val_acc": best, "test_acc": float((yt == pt).mean()),
"confusion_matrix": confusion_matrix(yt, pt).tolist(),
"per_class": classification_report(yt, pt, target_names=classes, output_dict=True),
}
json.dump(metrics, open(ROOT / "reports" / CFG["run"] / "metrics.json", "w"), indent=2)
print("test accuracy:", round(metrics["test_acc"], 3))
print(np.array(metrics["confusion_matrix"]))8Look at the mistakes (Day 5)
Most errors are data problems: a blurry photo, a wrong folder, a class that looks too similar. Fix the data, not the code.
import matplotlib.pyplot as plt
from PIL import Image
wrong = [i for i, (a, b) in enumerate(zip(yt, pt)) if a != b]
fig, axes = plt.subplots(1, max(1, min(6, len(wrong))), figsize=(14, 3), squeeze=False)
for ax, i in zip(axes[0], wrong[:6]):
path, _ = test_ds.samples[i]
ax.imshow(Image.open(path)); ax.set_title(f"true {classes[yt[i]]}\npred {classes[pt[i]]}"); ax.axis("off")
plt.show()Then move predict, the transforms and the training loop into sutra_core/train.py, and import them with %load_ext autoreload + %autoreload 2 at the top of the notebook.
Test Test it
These are two separate cells in two notebooks. Each notebook only knows its own variables: leaks and counts live in NB10, metrics lives in NB11.
assert not leaks, f"{len(leaks)} photos appear in more than one split: {leaks[:3]}"
assert all(len(list((OUT / "train" / c).iterdir())) >= 35 for c in counts), f"need at least 50 photos per class, have {counts} (a dry run stops here: expected)"
assert max(counts.values()) <= 1.5 * min(counts.values()), f"classes unbalanced {counts}: shoot more of the smallest class"
print("✅ dataset OK")assert metrics["test_acc"] >= 0.85, f"test accuracy {metrics['test_acc']:.3f}: add more varied photos or check labels"
print("✅ image model OK")Reproducibility check: Restart and Run All in NB11 a second time. The test accuracy should land within about 1 point of the first run.
metrics.json with a confusion matrix, and the second run agrees.Done Done when
- One option chosen (A: clean / stain / hole, or B: three part types)
- About 60 photos per class across all 4 light × background blocks, copied to the Jetson
- Clean 70/15/15 split with zero duplicates across splits
- best.pt and metrics.json saved
- Test accuracy ≥ 85%
- Misclassified photos reviewed and data fixed where needed
- Helpers moved into sutra_core/ and imported by the notebook
If it breaks
| Symptom | Fix |
|---|---|
UserWarning: Unable to import Axes3D when NB10 imports matplotlib | The venv can still see Ubuntu's old matplotlib. Do the one-time switch to an isolated venv in Step 02, substep 2, restart JupyterLab and the kernel. |
FileNotFoundError: Found no valid file for the classes … on the val or test line of NB11 | Too few photos per class reached NB10, so the 15% val/test share rounded down to zero. Usually: only a handful of photos copied, or the old NB10 skipped .JPG/.HEIC files. Count what's really there with the cell below this table, add photos (50+ per class), re-run the current NB10, then NB11. |
Warning: You are sending unauthenticated requests to the HF Hub in NB11 | Harmless: timm downloads the pretrained MobileNet from Hugging Face without logging in, and it still works (cached after the first run). To remove the warning and get higher download limits, log in once with a free Read token: hf auth login (Step 02, substep 5), then restart the kernel. |
NameError: name 'leaks' is not defined (or 'metrics') | The NB10 checks were pasted into NB11, or the other way round. The Test section has one cell for the end of NB10 and a separate cell for the end of NB11. |
AssertionError: … photos appear in more than one split | The same picture exists twice, e.g. copied from the phone twice (IMG_1234.jpg and IMG_1234 (1).jpg). The current NB10 skips exact duplicates inside a class and stops if one photo sits in two class folders, naming both files. Copy the current NB10, re-run it, then the leak cell prints any remaining pairs. |
NB10 shows extra classes, e.g. {'clean': 17, 'defect': 3, 'good': 5, 'hole': 16, 'stain': 17} | Old test folders are still in data/raw/images. Park them in data/raw/_parked (substep 4) and check CLASSES at the top of NB10. Then re-run NB10. |
| NB10 says "skipped … .heic" | iPhone HEIC files. Set Camera → Formats → Most Compatible and reshoot, or convert them to JPG on the laptop before copying. |
| 100% accuracy on the first try | Suspicious. Check that every class was shot in all 4 blocks; the model may be reading the background or light instead of the defect. Test it on a new cloth photographed in a new spot. |
stain and hole get mixed up | Defects too small or too far away. Shoot closer (15 cm) and make defects at least 1.5 cm across. View a photo at thumbnail size: if you can't see the defect, neither can the model. |
| Photos look sideways in the sample grid | Run the new NB10 again: ImageOps.exif_transpose applies the phone's rotation. |
| Accuracy stuck near chance | Check folder names and labels; open a few images per class; lower the learning rate to 3e-4. |
| Train accuracy high, test low | Overfitting: more varied photos, stronger augmentation, fewer epochs. |
| "Killed" or kernel dies | Lower batch to 16, set num_workers=0, close other kernels. |
| Pretrained weights won't download | Download once on the laptop (timm cache) and copy it, or check the Jetson's internet. |
| Grayscale X-ray images later (T3) | Add T.Grayscale(num_output_channels=3) first in both transforms. |
Count what each folder really holds, and which file types are in your raw folders:
from pathlib import Path
from collections import Counter
ROOT = Path.home() / "sutraai"
for c in sorted(p for p in (ROOT / "data/raw/images").iterdir() if p.is_dir()):
kinds = Counter(f.suffix for f in c.iterdir() if f.is_file())
print(f"raw {c.name:10} {sum(kinds.values()):4d} files {dict(kinds)}") # e.g. {'.jpg': 58, '.JPG': 2}
for split in ["train", "val", "test"]:
d = ROOT / "data/processed/images" / split
if d.exists():
print(split.ljust(5), {c.name: len(list(c.iterdir())) for c in sorted(d.iterdir())})Learn more
- ZTM "PyTorch for Deep Learning": computer vision, custom datasets and transfer learning sections (course page)
- learnpytorch.io: chapters 03–06 (free)
- Free: NVIDIA DLI "Getting Started With AI on Jetson Nano" (image classification on Jetson), on NVIDIA AI Learning Essentials