# SutraAI — Jetson Orin NX Beginner Guide (Notebook-First)

**Who this is for:** an experienced software developer who is **new to AI models**. You know Linux, Python and web well. This guide teaches the AI side step by step, on your **Jetson Orin NX 16GB (Super dev kit) running JetPack 6.2.1 with CUDA 12.6.68**, using **Jupyter notebooks only** for the first 3 weeks. There are no APIs or web UI until Week 4.

**Your board (from jtop 7.2.1):**
- Jetson Orin NX 16GB on the dev kit carrier (P3767-0000), Super config
- JetPack 6.2.1, L4T 36.4.7, Ubuntu 22.04, kernel 5.15.148-tegra
- CUDA 12.6.68, cuDNN 9.3.0, TensorRT 10.3.0.30, VPI 3.2.4.0
- Python 3.10.12, OpenCV 4.8.0 (no CUDA), CUDA arch 8.7
- Hostname `jetson`, LAN `<jetson-ip>`, Tailscale and Docker set up

"7.2.1" in the jtop header is the version of the jtop tool (jetson-stats), not JetPack. Your JetPack is 6.2.1.

**Rule for Weeks 1–3:** if a step isn't working and verified in a notebook, it doesn't go into FastAPI or Next.js.

**Companion files:**
- `SutraAI_30_Day_Plan.md` says **when** to do each step. This guide says **how**.
- `SutraAI_Solutions_Catalog.md` is the product map.

---

## Contents

1. [Know your Jetson](#1-know-your-jetson-orin-nx-16gb)
2. [Choose and install JetPack](#2-choose-and-install-jetpack)
3. [First-boot checklist](#3-first-boot-checklist-day-1)
4. [CUDA libraries explained](#4-cuda-libraries-explained-what-each-one-does)
5. [Python environment and JupyterLab](#5-python-environment-and-jupyterlab-day-2)
6. [Project folder and notebook rules](#6-project-folder-and-notebook-rules)
7. [The notebook path (NB00 → NB90)](#7-the-notebook-path-nb00--nb90)
8. [NVIDIA models to test](#8-nvidia-models-to-test-on-your-jetson)
9. [Memory budget for 16 GB](#9-memory-budget-for-16-gb)
10. [The adoption gate (before any web code)](#10-the-adoption-gate-before-any-web-code)
11. [Courses: Udemy and free](#11-courses-udemy-and-free)
12. [Troubleshooting](#12-troubleshooting)
13. [Glossary for beginners](#13-glossary-for-beginners)

---

## 1. Know Your Jetson Orin NX 16GB

| Part | What it means for you |
|---|---|
| **GPU:** NVIDIA Ampere, 1024 CUDA cores + 32 Tensor Cores, CUDA arch 8.7 | Trains small models (MobileNet, 1D-CNN) in minutes. Runs 4B LLMs easily and can try 9B at 4-bit. |
| **DLA:** deep-learning accelerators (the Orin Nano has none) | An extra, low-power TensorRT target to try in NB12 |
| **CPU:** 8-core Arm Cortex-A78AE | Data loading, audio processing (MFCC), TFLite conversion; use `num_workers=4` |
| **Memory:** 16 GB, **shared by CPU and GPU** | There is no separate GPU memory. 16 GB is enough for an LLM next to a small training job, but not for everything at once. |
| **Storage:** **NVMe SSD** with 100 GB+ free | Datasets, model files (GGUF LLMs are 2–6 GB each) and swap |
| **Power modes** (`nvpmodel`) | Super config: 10 W, 15 W, 25 W, 40 W, MAXN SUPER. Yours was at 15 W; use MAXN SUPER (`sudo nvpmodel -m 0`) for training. |

**Hardware checklist before Day 1:**
- Orin NX 16GB dev kit (Super config) with its **original power supply** and a working fan
- NVMe SSD with 100 GB+ free
- Ethernet or Wi-Fi (you have Ethernet at <jetson-ip>, plus Tailscale)
- A laptop for SSH
- Optional: USB microphone (bird sounds), USB webcam (drowsiness), and your phone (sensor data)

---

## 2. Choose and Install JetPack

JetPack is NVIDIA's OS + driver + AI library bundle for Jetson. **First check what you already have:**

```bash
cat /etc/nv_tegra_release        # R36.x = JetPack 6.x   ·   R39.x = JetPack 7.2.x
apt show nvidia-jetpack 2>/dev/null | grep Version
```

| | **JetPack 6.2.x** (latest 6.2.2) | **JetPack 7.2.x** |
|---|---|---|
| Ubuntu / Python | 22.04 / 3.10 | 24.04 / 3.12 |
| CUDA · cuDNN · TensorRT | 12.6 · 9.x · 10.x | 13.2 · 9.20 · 10.16 |
| Tutorials and containers | **Most tutorials and jetson-containers images target this** | Newer. Some CUDA 12.6-based containers fail on the CUDA 13.2 driver, so you may need to build tools like llama.cpp from source. |
| Install method | SDK Manager (Ubuntu 22.04 host) or SD-card image | Unified ISO on a USB stick, or SDK Manager (Ubuntu 24.04 host). There is no SD-card image. |
| **Recommendation** | **Best for a beginner doing this 3-week notebook phase** | Fine if it's already installed. Follow the JetPack 7.2 notes in this guide. |

**Your board: JetPack 6.2.1 (L4T 36.4.7), CUDA 12.6.68. Keep it.** Everything in this guide and in the NVIDIA-AI HTML pages is written for this version. You don't need to flash anything.

- JetPack 7.2 would change CUDA to 13.2 and Python to 3.12, so the PyTorch wheels, the venv and some containers would all change. Don't upgrade during the 3-week phase.

**Only if you ever need to re-flash (SDK Manager):**
1. Put the board in **recovery mode**: connect pins 9–10 (FC REC ↔ GND) on the button header, then apply power. On the host, `lsusb | grep -i nvidia` should show the device.
2. In SDK Manager, pick **Jetson Orin NX 16GB** on the developer kit carrier with the **Super** configuration, storage **NVMe**, and **fill in the OS pre-config** (username, password, hostname). If you skip pre-config, the first boot waits on a setup wizard you can't reach headless.
3. Flash the OS first, then install the SDK components (CUDA, cuDNN, TensorRT) in a second pass.

> Don't do JetPack upgrades in the middle of the 3-week phase. Pick one version and keep it.

---

## 3. First-Boot Checklist (Day 1)

Run these over SSH from your laptop: `ssh <user>@<jetson-ip>` (or your Tailscale address when away from home).

```bash
# 0) System up to date + make sure all AI libraries are installed
sudo apt update && sudo apt upgrade -y
sudo apt install -y nvidia-jetpack              # CUDA, cuDNN, TensorRT, VPI (skip if already installed)

# 1) Max performance mode: yours was at 15 W. MAXN SUPER is mode 0 on Orin NX (or use jtop → 6 CTRL)
sudo nvpmodel -q --verbose
sudo nvpmodel -m 0
sudo nvpmodel -q

# 2) Optional on 16 GB: run headless, frees roughly 0.5–1 GB of RAM (undo with graphical.target)
sudo systemctl set-default multi-user.target

# 3) 16 GB swap on NVMe (safety net when memory runs out; it's slow, not a fix)
sudo swapoff -a
sudo fallocate -l 16G /swapfile && sudo chmod 600 /swapfile
sudo mkswap /swapfile && sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
# Optional on JetPack 6: disable zram (compressed RAM swap) so the NVMe swap is used instead
sudo systemctl disable nvzramconfig

# 4) jtop (already installed: jtop 7.2.1). Keep it updated
sudo pip3 install -U jetson-stats
sudo reboot
```

After the reboot, run `jtop`. The title bar should show MAXN SUPER instead of 15W, about 15 GB of RAM, 16 GB of swap, all 8 CPU cores and the GPU.

- Use `sudo jetson_clocks` only while benchmarking. It pins clocks to maximum, which means more heat.

**Daily habits for a healthy Jetson:**
- Shut down with `sudo shutdown now`. Never just pull the power.
- Keep jtop open in a second terminal while training. Watch RAM and temperature.
- Keep at least 30 GB free on the NVMe.
- Push notebooks to git every day. Use `nbstripout` so notebook outputs don't bloat the repo.

---

## 4. CUDA Libraries Explained: What Each One Does

You don't call most of these directly. PyTorch, TensorRT and ONNX Runtime use them for you. Knowing what they are helps you debug.

| Library | Plain-English job | Where SutraAI uses it | How to check it |
|---|---|---|---|
| **CUDA Toolkit** (runtime + `nvcc` compiler) | Lets programs run code on the GPU | Everything GPU; building llama.cpp | `nvcc --version` (add CUDA to PATH first, below) |
| **cuDNN** | Fast building blocks for neural networks (convolutions, etc.) | PyTorch training | `python -c "import torch; print(torch.backends.cudnn.version())"` |
| **cuBLAS** | Fast matrix math | Inside PyTorch and llama.cpp | Comes with CUDA |
| **cuFFT** | FFT on the GPU | Optional speed-up for MFCC later (NumPy on CPU is fine for now) | Comes with CUDA |
| **TensorRT** | Optimizes a trained model into a fast **engine** for this exact GPU | Jetson deploy target (`trtexec`) | `/usr/src/tensorrt/bin/trtexec --help` · `python3 -c "import tensorrt; print(tensorrt.__version__)"` |
| **VPI** | Hardware-accelerated image processing | Not needed now (maybe for camera pipelines later) | `dpkg -l \| grep vpi` |
| **DeepStream** | Multi-camera video analytics pipelines | Months 3+ (not needed now) | — |
| **NVIDIA Container Toolkit** | Lets Docker containers use the GPU | Optional containers (Section 5, option B) | `docker info \| grep -i runtime` |

**Add CUDA to your PATH** (one time):

```bash
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc && nvcc --version
```

---

## 5. Python Environment and JupyterLab (Day 2)

**Option A (recommended): a native virtual environment.** It's simple and transparent, which suits notebooks.

### Your board: JetPack 6.2.1, CUDA 12.6.68, Python 3.10.12

| Package | Version | From |
|---|---|---|
| torch | **2.8.0** | Jetson AI Lab `jp6/cu126` (cp310) |
| torchvision | **0.23.0** | Jetson AI Lab `jp6/cu126` |
| torchaudio | **2.8.0** | Jetson AI Lab `jp6/cu126` |
| onnxruntime-gpu | **1.23.0** | Jetson AI Lab `jp6/cu126` |
| numpy | **1.26.4** | PyPI |
| matplotlib · pillow · scipy | **3.10.9 · 11.3.0 · 1.15.3** | PyPI |
| TensorRT (Python) | 10.3.0.30 | JetPack (system), linked into the isolated venv |
| cuDSS (system library) | 0.7.1.4, CUDA 12 | NVIDIA download (torch 2.8.0 needs `libcudss.so.0`) |
| cuSPARSELt (system library) | 0.6.3.2 | NVIDIA download (torch 2.8.0 needs `libcusparseLt.so.0`) |

torch 2.8.0 is the version NVIDIA forum users confirmed on JetPack 6.2. The index also has newer sets (2.9.1, 2.10.0, 2.11.0); stay on 2.8.0 for the 3-week notebook phase.

**The Jetson torch 2.8.0 wheel needs two NVIDIA libraries that JetPack 6.2.1 doesn't install.** Without them pip finishes fine, but `import torch` fails with `ImportError: libcudss.so.0: cannot open shared object file` (or the same for `libcusparseLt.so.0`). Install them once, before torch:

```bash
sudo apt-get update
sudo apt-get install -y python3-venv python3-pip libopenblas-dev libomp-dev ffmpeg git

mkdir -p ~/downloads && cd ~/downloads
# cuDSS 0.7.1 for Jetson (CUDA 12) → libcudss.so.0
wget https://developer.download.nvidia.com/compute/cudss/redist/libcudss/linux-aarch64/libcudss-linux-aarch64-0.7.1.4_cuda12-archive.tar.xz
echo "f283c31b6badf4a5277014295705eac8e6e28f27de30d746719ea1cef7a750b8  libcudss-linux-aarch64-0.7.1.4_cuda12-archive.tar.xz" | sha256sum -c
sudo mkdir -p /opt/nvidia
sudo tar xf libcudss-linux-aarch64-0.7.1.4_cuda12-archive.tar.xz -C /opt/nvidia
dirname "$(find /opt/nvidia/libcudss-linux-aarch64-0.7.1.4_cuda12-archive -name 'libcudss.so.0' | head -n1)" | sudo tee /etc/ld.so.conf.d/cudss.conf
# cuSPARSELt 0.6.3 for Jetson → libcusparseLt.so.0
wget https://developer.download.nvidia.com/compute/cusparselt/redist/libcusparse_lt/linux-aarch64/libcusparse_lt-linux-aarch64-0.6.3.2-archive.tar.xz
echo "91501d0c05d1ff0dd67399ecd7c1bf76a620e842dce54ae4c8a1f07cec0673e5  libcusparse_lt-linux-aarch64-0.6.3.2-archive.tar.xz" | sha256sum -c
sudo tar xf libcusparse_lt-linux-aarch64-0.6.3.2-archive.tar.xz -C /opt/nvidia
dirname "$(find /opt/nvidia/libcusparse_lt-linux-aarch64-0.6.3.2-archive -name 'libcusparseLt.so.0' | head -n1)" | sudo tee /etc/ld.so.conf.d/cusparselt.conf
sudo ldconfig
ldconfig -p | grep -E "libcudss.so.0|libcusparseLt.so.0|libopenblas.so.0"   # expect 3+ lines
```

Then the venv and the pinned packages:

```bash
mkdir -p ~/sutraai && cd ~/sutraai
python3 -m venv .venv                             # isolated: Ubuntu's old Python packages stay out

# Version lock: every pip install in this venv must keep these versions
cat > constraints-jp621.txt <<'EOF'
torch==2.8.0
torchvision==0.23.0
torchaudio==2.8.0
onnxruntime-gpu==1.23.0
numpy==1.26.4
matplotlib==3.10.9
pillow==11.3.0
scipy==1.15.3
EOF
echo 'export PIP_CONSTRAINT=$HOME/sutraai/constraints-jp621.txt' >> .venv/bin/activate
source .venv/bin/activate
pip install --upgrade pip
pip install numpy==1.26.4

# Link only JetPack's TensorRT 10.3 Python bindings into the venv
TRT_DIR=$(/usr/bin/python3 -c "import tensorrt, os; print(os.path.dirname(os.path.dirname(tensorrt.__file__)))")
SP="$VIRTUAL_ENV/lib/python3.10/site-packages"
for p in "$TRT_DIR"/tensorrt*; do ln -sfn "$p" "$SP/$(basename "$p")"; done
python -c "import tensorrt; print(tensorrt.__version__)"                       # expect: 10.3.0.30

# GPU wheels built for JetPack 6 / CUDA 12.6 from the Jetson AI Lab index
pip install --no-cache-dir torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://pypi.jetson-ai-lab.io/jp6/cu126/+simple/
pip install --no-cache-dir onnxruntime-gpu==1.23.0 --index-url https://pypi.jetson-ai-lab.io/jp6/cu126/+simple/

# Any library torch links against that the system can't find? (doesn't import torch)
TORCH_LIB=$(python -c "import importlib.util, pathlib; print(pathlib.Path(importlib.util.find_spec('torch').origin).parent / 'lib')")
for f in "$TORCH_LIB"/*.so*; do ldd "$f"; done | grep "not found" | sort -u      # expect: no output

# Everything else from normal PyPI (the lock keeps torch and numpy in place)
pip install jupyterlab ipykernel ipywidgets matplotlib pandas scikit-learn tqdm pyyaml \
            timm onnx librosa soundfile openai sentence-transformers thop nbstripout ai-edge-litert
python -m ipykernel install --user --name sutraai --display-name "SutraAI (venv · torch 2.8 · CUDA 12.6)"
pip check
python -c "import sys; print([p for p in sys.path if 'dist-packages' in p])"   # expect: [] (nothing from Ubuntu)
```

**Why an isolated venv:** Ubuntu keeps old Python packages in `/usr/lib/python3/dist-packages` (matplotlib 3.5, numpy 1.21, scipy, Pillow). A venv made with `--system-site-packages` sees them, pip skips them as "already installed", and you get half-new, half-old libraries (`'RcParams' object has no attribute '_get'`, `Unable to import Axes3D`). The isolated venv sees none of them; TensorRT, the only system package needed, is linked in by name.

**Already made the venv with `--system-site-packages`?** Switch it instead of starting over: `sed -i 's/^include-system-site-packages = .*/include-system-site-packages = false/' .venv/pyvenv.cfg`, link TensorRT as above, re-run the two torch/onnxruntime `pip install` lines and the toolkit line (they only add what's missing), then `pip check`. Full commands: Step 02 page, substep 2.

> **Always verify** after installing: `python -c "import torch, torchvision, numpy; print(torch.__version__, torchvision.__version__, numpy.__version__, torch.version.cuda, torch.cuda.is_available())"` must print `2.8.0 0.23.0 1.26.4 12.6 True`.
>
> If the last value is `False`, pip pulled a CPU-only wheel from normal PyPI. Run `pip uninstall -y torch torchvision torchaudio`, then reinstall the pinned versions from the Jetson index with `--no-cache-dir`.
>
> **onnx2tf** (int8 TFLite in NB12) is version 2.6.9: it no longer needs TensorFlow, but it needs Python 3.12+. Run it in Google Colab, or in a separate Python 3.12 venv made with uv (`uv venv --python 3.12 ~/sutraai/.venv-convert`). Never install it into the main venv.

If you ever upgrade to JetPack 7.2 (CUDA 13, Python 3.12), this whole package set changes. Recreate the venv against the matching index instead of mixing versions.

### Open JupyterLab Securely from Your Laptop

```bash
# On the Jetson (inside the venv)
cd ~/sutraai && jupyter lab --no-browser --port 8888

# On your laptop: an SSH tunnel, so nothing is exposed on your network
ssh -N -L 8888:localhost:8888 <user>@<jetson-ip>
# then open http://localhost:8888 and paste the token printed on the Jetson
```

Alternative: **VS Code → Remote-SSH → open `~/sutraai` → pick the "SutraAI (venv)" kernel.** This is great if you already live in VS Code.

**Option B (later): containers.** NGC / jetson-containers images come with PyTorch preinstalled. They're useful for Nemotron and NanoOWL tutorials on JetPack 6. On 7.2, check that the image matches CUDA 13.

---

## 6. Project Folder and Notebook Rules

```
~/sutraai/
├── notebooks/            # NB00 … NB90 (one notebook = one step of the flow)
├── sutra_core/          # .py helpers the notebooks import (becomes ml_core in Week 4)
│   ├── data.py  dsp.py  models.py  train.py  export.py  profile.py  device_sim.py
├── configs/              # pipeline.yaml files (image.yaml, sensor.yaml, audio.yaml, detect.yaml)
├── data/raw/  data/processed/
├── models/               # best.pt checkpoints, GGUF LLMs
├── exports/              # .onnx  .tflite  .engine  model_data.cc  manifest.json
├── reports/              # metrics.json, confusion-matrix PNGs, profile tables
└── BUILDLOG.md
```

**Every notebook has the same 7 cells:**
1. **Goal** (markdown): what this notebook proves
2. **Config**: paths, seed, hyper-parameters
3. **Imports** + `%load_ext autoreload` / `%autoreload 2`
4. **Steps**: small cells with a plot or print after each
5. **Results**: metrics and plots
6. **✅ Checks**: `assert` statements (e.g., `assert test_acc > 0.80`)
7. **Save**: artifacts to `models/`, `exports/`, `reports/`

**Rules:**
- Before ticking a day done, do **Kernel → Restart & Run All**. If it doesn't run top-to-bottom, it isn't done.
- From Week 2, move any function you use twice into `sutra_core/*.py`. That makes the Week 4 move to FastAPI a copy, not a rewrite.
- Set seeds (`random`, `numpy`, `torch`) so results repeat within about 1%.
- Run one heavy kernel at a time. Shut down notebooks you're not using (Running Terminals and Kernels panel).

---

## 7. The Notebook Path (NB00 → NB90)

```
Week 1  NB00 env → NB01 GPU hello → NB02 ML basics → NB10 image data → NB11 image train → NB12 export + validate
Week 2  NB20 sensor data → NB21 sensor train (CNN + Transformer) → NB30 audio data → NB31 MFCC from scratch → NB32 audio train
        NB90 v0 (pipeline.yaml runner)
Week 3  NB40 detection data → NB41 detection train → NB42 detection export → NB50 profiler → NB51 device simulator
        NB60 Nemotron advisor → NB61 RAG basics → NB62 NVIDIA vision helpers → NB90 final (adoption gate)
```

### NB00: Environment Check
**Goal:** prove every library sees the GPU.

```python
import platform, torch, onnxruntime as ort
print("Python", platform.python_version())
print("torch", torch.__version__, "| CUDA available:", torch.cuda.is_available())
print("CUDA (torch build):", torch.version.cuda, "| cuDNN:", torch.backends.cudnn.version())
print("GPU:", torch.cuda.get_device_name(0))
free, total = torch.cuda.mem_get_info(); print(f"Memory visible to GPU: {free/1e9:.1f} free / {total/1e9:.1f} GB")
try:
    import tensorrt; print("TensorRT", tensorrt.__version__)
except ImportError:
    print("TensorRT not visible: run the TensorRT link lines in §5")
print("ONNX Runtime providers:", ort.get_available_providers())
```
✅ **Check:** CUDA available is True; the providers list includes `CUDAExecutionProvider` (and ideally `TensorrtExecutionProvider`).

### NB01: GPU Hello
**Goal:** feel the GPU speed-up and learn what a tensor is.

```python
import torch, time
def bench(device, n=2048, reps=10):
    a, b = torch.randn(n, n, device=device), torch.randn(n, n, device=device)
    for _ in range(2): a @ b                        # warm-up
    if device == "cuda": torch.cuda.synchronize()
    t = time.perf_counter()
    for _ in range(reps): a @ b
    if device == "cuda": torch.cuda.synchronize()
    return (time.perf_counter() - t) / reps * 1000
print(f"CPU {bench('cpu'):.1f} ms  vs  GPU {bench('cuda'):.1f} ms  per 2048×2048 matrix multiply")
```
✅ **Check:** the GPU is clearly faster. Also try `float16` on the GPU and note the difference.

### NB02: ML Basics (Toy Dataset)
**Goal:** understand the training loop before using your own data.
1. Load a small FashionMNIST subset with `torchvision.datasets`.
2. Split into train, validation and test.
3. Train a tiny network for 10 epochs and plot **train vs validation loss**.
4. Deliberately overfit: tiny data, many epochs. See validation loss rise.
5. Fix it with augmentation or early stopping.

✅ **Check:** you can explain in your own words what epoch, batch, loss, overfitting and the test set are (Glossary).

### NB10: Image Dataset (Product Defect QC, Template T5)
**Goal:** a clean, balanced dataset on disk.
- Photograph small objects, not people or furniture. Recommended: **fabric QC** with classes `clean` / `stain` / `hole`, using 15 cloth pieces of 20×20 cm (the same pieces for all classes; 12+ different stains and holes, each at least 1.5 cm). Alternative: 3 part types (`screw` / `nut` / `washer`).
- Take **60 phone photos per class**: 15 in each of 4 blocks (daylight or LED light × two different backgrounds). Every block must include every class, or the model learns the background instead of the defect. Normal photo mode, 1× lens, JPG (iPhone: Formats → Most Compatible), no WhatsApp transfers.
- Copy one folder per class to `~/sutraai/data/raw/images/<class>/`. NB10 makes 512 px copies, fixes rotation, and splits 70/15/15.
- Layout: `data/raw/images/<label>/*.jpg`. Split 70/15/15 into `data/processed/{train,val,test}/<label>/`.
- Plot a grid of samples and a class-count bar chart.

✅ **Check:** every class has at least 40 training images, and no image appears in two splits (hash check).

### NB11: Image Training (Transfer Learning)
**Goal:** your first real model.
- `timm.create_model("mobilenetv2_050", pretrained=True, num_classes=N)` at 96×96 for MCU targets; try 160×160 for Jetson.
- Augmentation (random crop, flip, colour jitter), AdamW, mixed precision (`torch.autocast`), 30 epochs.
- Save `models/image_v1/best.pt` and `reports/image_v1/metrics.json` (val/test accuracy, per class, confusion matrix).
- Look at the **misclassified images**. Most errors are data problems.

✅ **Check:** test accuracy ≥ 85% on your data, and Restart & Run All reproduces it within ±1%.

### NB12: Export and Validate
**Goal:** prove the model survives conversion, which is the heart of edge AI.
1. **ONNX:** `torch.onnx.export(...)`. Compare PyTorch vs ONNX Runtime outputs on 20 images (max difference < 1e-3).
2. **TensorRT:** `!/usr/src/tensorrt/bin/trtexec --onnx=exports/image_v1.onnx --saveEngine=exports/image_v1_fp16.engine --fp16`, then record latency.
3. **TFLite int8:** convert with onnx2tf 2.6.9 and real calibration images: `onnx2tf -i model.onnx -o out -oiqt -cind input calib.npy "[[[[0.485,0.456,0.406]]]]" "[[[[0.229,0.224,0.225]]]]"`. It needs Python 3.12+, so run it in Google Colab (recommended) or in a Python 3.12 `.venv-convert` made with `uv venv --python 3.12`. Use the `*_full_integer_quant.tflite` file.
4. **int8 accuracy check** with LiteRT. **Target: less than a 3-point drop vs float32.**
5. **C array:** `!xxd -i exports/image_v1_int8.tflite > exports/image_v1_model_data.cc`.

```python
import numpy as np
from ai_edge_litert.interpreter import Interpreter
def tflite_accuracy(path, X, y):     # full-integer model; X already in the model's layout (images: NHWC, float 0–1 or normalized)
    itp = Interpreter(model_path=str(path)); itp.allocate_tensors()
    inp, out = itp.get_input_details()[0], itp.get_output_details()[0]
    scale, zero = inp["quantization"]
    hits = 0
    for xi, yi in zip(X, y):
        q = np.clip(np.round(xi / scale + zero), -128, 127).astype(inp["dtype"])
        itp.set_tensor(inp["index"], q[None]); itp.invoke()
        hits += int(itp.get_tensor(out["index"])[0].argmax() == yi)
    return hits / len(y)
```
> The Jetson's Python is 3.10, and the last onnx2tf for 3.10 (1.29.x) has to compile several packages from source on ARM. Run the conversion in Google Colab instead (upload the `.onnx` and `calib.npy`, download the `.tflite`). It's CPU-only, and the rest of the notebook stays on the Jetson.

✅ **Check:** the ONNX difference is tiny, the TensorRT engine runs, and the int8 drop is under 3 points.

### NB20: Sensor Data (Fall Detection T4 / Machine Vibration T6)
**Goal:** turn raw accelerometer recordings into training windows.
- Record with the free **phyphox** phone app (accelerometer, export CSV). For falls, drop the phone onto a mattress inside a pouch, plus normal walking, sitting and stairs. For vibration, tape the phone to a fan at different speeds.
- Load the CSV → resample to 50 Hz → plot the 3 axes → cut into **windows** (e.g., 4 s window, 1 s stride) → label → split **by recording**, not by window, so windows from the same recording don't leak between train and test.

✅ **Check:** the plots show visibly different patterns per class, and the train and test splits come from different recordings.

### NB21: Sensor Training (1D-CNN vs Tiny Transformer)
**Goal:** compare two architectures honestly.
- 1D-CNN on raw windows, and a tiny Transformer (d_model 32, 2 heads, 2 layers).
- Same splits, seeds and epochs. Make a table of accuracy, parameter count and int8 size.
- Export both to int8 with the NB12 helpers (now in `sutra_core/export.py`). Check that the Transformer converts cleanly to TFLite. If it doesn't, the 1D-CNN is the official model.

✅ **Check:** both models trained; there's an int8 model with a < 3-point drop; the decision is written in BUILDLOG.

### NB30: Audio Data (Bird Sound T1)
**Goal:** clean 16 kHz mono clips.
- Record birds and background noise (traffic, wind, people) with your phone. Convert: `ffmpeg -i in.m4a -ac 1 -ar 16000 out.wav`.
- Plot the waveform and spectrogram. Cut into 1 s windows with 0.5 s stride. Keep a separate "noise" class.

✅ **Check:** at least 15 minutes per class, and spectrograms look different per class.

### NB31: MFCC from Scratch (Your Embedded Skills Matter Here)
**Goal:** understand and own the DSP, because the chip must run the same maths in Month 3.

```python
import numpy as np
hz_to_mel = lambda f: 2595 * np.log10(1 + f / 700)
mel_to_hz = lambda m: 700 * (10 ** (m / 2595) - 1)

def mel_filterbank(sr, n_fft, n_mels):
    mels = np.linspace(hz_to_mel(0), hz_to_mel(sr / 2), n_mels + 2)
    bins = np.floor((n_fft + 1) * mel_to_hz(mels) / sr).astype(int)
    fb = np.zeros((n_mels, n_fft // 2 + 1))
    for m in range(1, n_mels + 1):
        l, c, r = bins[m - 1], bins[m], bins[m + 1]
        if c > l: fb[m - 1, l:c] = (np.arange(l, c) - l) / (c - l)
        if r > c: fb[m - 1, c:r] = (r - np.arange(c, r)) / (r - c)
    return fb

def mfcc(x, sr=16000, frame_ms=20, stride_ms=20, n_fft=512, n_mels=32, n_coef=13):
    x = np.append(x[0], x[1:] - 0.97 * x[:-1])                                  # 1 pre-emphasis
    flen, step = sr * frame_ms // 1000, sr * stride_ms // 1000
    n = 1 + (len(x) - flen) // step
    frames = x[np.arange(flen)[None, :] + step * np.arange(n)[:, None]] * np.hamming(flen)  # 2 frames + window
    power = np.abs(np.fft.rfft(frames, n_fft)) ** 2 / n_fft                      # 3 power spectrum
    logmel = np.log(power @ mel_filterbank(sr, n_fft, n_mels).T + 1e-10)         # 4 log mel energies
    k = np.arange(n_mels)
    dct = np.cos(np.pi / n_mels * (k[None, :] + 0.5) * np.arange(n_coef)[:, None])  # 5 DCT-II
    return logmel @ dct.T                                                         # shape: [frames, n_coef]
# 1 s at 16 kHz -> 50 frames × 13 coefficients
```
- Plot the output of each step: this is the best way to learn MFCC.
- Compare with `librosa.feature.mfcc`. It won't match exactly, because librosa uses a different mel formula and dB scaling by default. Check that the patterns correlate (> 0.9). **Exact matching only matters between this Python and your future C code.**
- Save **golden vectors** to `exports/golden/mfcc_*.npz` (input window + expected output). In Month 3 the C/CMSIS-DSP version must reproduce them.

✅ **Check:** the shape is (50, 13) for 1 s, and golden vectors are saved.

### NB32: Audio Training and Export
**Goal:** the bird classifier.
- 1D-CNN on MFCC. Add augmentation: mix in background noise and shift in time.
- Confusion matrix: which bird gets confused with noise? Then export to int8 and check accuracy.

✅ **Check:** test accuracy ≥ 80% on your recordings, and int8 drop < 3 points.

### NB90 v0 (End of Week 2): The `pipeline.yaml` Runner
**Goal:** one config drives everything, which is the seed of the platform.
- Write `configs/image.yaml`, `sensor.yaml` and `audio.yaml` (format in the 30-day plan).
- NB90 reads a YAML file → runs data → DSP → train → evaluate → export → saves reports. No hand edits.

✅ **Check:** all three configs run from a clean kernel.

### NB40: Detection Data (Driver Drowsiness T2)
**Goal:** a box-labelled dataset.
- Webcam photos of eyes open / closed and yawn / no yawn. **Get consent from anyone photographed.** Faces are personal data.
- Label boxes with **Label Studio** (`pip install label-studio` on your laptop) and export in **YOLO format**. Optionally pre-label with NanoOWL (NB62) and just correct the boxes.

✅ **Check:** at least 200 labelled images, and every class has at least 100 boxes.

### NB41: Detection Training (Centroid Detector)
**Goal:** find where objects are and count them, small enough for an MCU.
- Use the `CentroidDetector` from the 30-day plan: a MobileNetV2 backbone with a stride-8 per-cell class head.
- Targets: the grid cell containing each box centre gets that class; every other cell is background. Weight object cells higher in the loss.
- Metric: **F1** of predicted centres vs true centres. Save overlay images with dots on detected objects.

✅ **Check:** F1 ≥ 0.75 per class, and the overlays look right.

### NB42: Detection Export
Use the same export helpers as before: ONNX → TensorRT and TFLite int8, then check F1 again after int8.

✅ **Check:** F1 drops by less than 0.05 after int8.

### NB50: Profiler
**Goal:** answer "will it fit on chip X?" for every model.
- **Flash:** model size plus a per-target runtime allowance.
- **Peak RAM:** largest live activation memory. This is an estimate.
- **Latency:** measured on the Jetson (ONNX Runtime / TensorRT); estimated for MCUs from operation counts. Label MCU numbers **"estimated"**.
- Output one table: models × targets (ESP32-S3, xG24, Cortex-M4F, Cortex-M7, Jetson, CPU).

✅ **Check:** the table is saved to `reports/profile.md`.

### NB51: Device Simulator (the Whole Flow, No API)
**Goal:** prove the device side of the product with no server and no chip.

```python
import json, hashlib, numpy as np
from ai_edge_litert.interpreter import Interpreter

class FakeDevice:
    """Acts like an ESP32: loads a model from a manifest, checks it, runs it, logs telemetry."""
    def __init__(self, manifest_path):
        self.telemetry = []
        self.install(manifest_path)
    def install(self, manifest_path):                       # "OTA update" = load a new manifest
        m = json.load(open(manifest_path))
        blob = open(m["artifact_path"], "rb").read()
        assert hashlib.sha256(blob).hexdigest() == m["sha256"], "corrupt download"
        self.m, self.itp = m, Interpreter(model_content=blob)
        self.itp.allocate_tensors()
        self.inp, self.out = self.itp.get_input_details()[0], self.itp.get_output_details()[0]
    def infer(self, features):                              # features = output of the same DSP as training
        s, z = self.inp["quantization"]
        q = np.clip(np.round(features / s + z), -128, 127).astype(np.int8)
        self.itp.set_tensor(self.inp["index"], q.reshape(self.inp["shape"])); self.itp.invoke()
        so, zo = self.out["quantization"]
        probs = (self.itp.get_tensor(self.out["index"])[0].astype(np.float32) - zo) * so
        label, conf = self.m["labels"][int(probs.argmax())], float(probs.max())
        self.telemetry.append({"model_version": self.m["version"], "label": label, "confidence": conf})
        return label, conf
```
Run a full story in the notebook:
1. Install v1.
2. Stream 200 test windows.
3. Collect low-confidence ones into a "review list".
4. Retrain with them (NB90).
5. Install v2 via a new manifest.
6. Show that accuracy went up.

✅ **Check:** v1 → v2 "OTA" works, the sha256 check catches a corrupted file, and the telemetry JSON is saved.

### NB60: Nemotron Training Advisor
**Goal:** a local LLM explains results in plain language (Section 8 shows how to run the model).

```python
from openai import OpenAI
import json
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")
metrics = json.load(open("reports/audio_v1/metrics.json"))
prompt = ("You are a training advisor for small edge-AI models. Using ONLY the numbers below, "
          "explain the results to a beginner in 5 short bullets and suggest the 2 most useful next steps.\n"
          + json.dumps(metrics))
reply = client.chat.completions.create(model="nemotron", temperature=0.2,
                                       messages=[{"role": "user", "content": prompt}])
print(reply.choices[0].message.content)
```
✅ **Check:** the advice refers to real numbers from `metrics.json` and invents none. Also record tokens/s and RAM from jtop.

### NB61: RAG Basics
**Goal:** answer "how do I…" questions from your own notes, with sources.
- Chunk `BUILDLOG.md` + `PIPELINE.md` into 300–500-token pieces → embed → cosine top-k → ask Nemotron to answer **only from those chunks** and cite them.
- Start with a small embedder (`BAAI/bge-small-en-v1.5` via sentence-transformers). Then try NVIDIA's `llama-nemotron-embed-1b-v2` from its Hugging Face model card and compare.
- Write 10 test questions with known answers and measure how often the right chunk is in the top 4 (hit@4).

✅ **Check:** hit@4 ≥ 8/10, and answers cite chunks.

### NB62: NVIDIA Vision Helpers (Optional but Valuable)
**Goal:** see whether NVIDIA's vision models can speed up labeling (Section 8).
- **Cosmos Reason 2 2B:** ask it about your images ("Are this person's eyes closed? Answer yes/no") and flag disagreements with your labels. These are likely labeling mistakes.
- **NanoOWL:** zero-shot boxes from a text prompt ("an eye", "an open mouth") to pre-label NB40 images.

✅ **Check:** a short table of time saved and error rate. Decide whether auto-labeling goes into Month 2.

### NB90 Final (Day 21): the Adoption Gate
All 4 references run from YAML, end to end, from a clean kernel. See Section 10.

---

## 8. NVIDIA Models to Test on Your Jetson

| Model | Type | Run with | What it does for SutraAI | On your 16 GB |
|---|---|---|---|---|
| **Nemotron 3 Nano 4B** (Q4_K_M GGUF) | LLM | llama.cpp `llama-server` | Training Advisor, RAG answers, later the in-app assistant | Yes, even next to a small training job |
| **Cosmos Reason 2 2B** (Q8_0 GGUF) | Vision-language model (image/video + text) | llama.cpp | Check labels, describe images, flag anomalies | Yes, can run next to Nemotron 4B |
| **llama-nemotron-embed-1b-v2** | Text embeddings | Python (Hugging Face) | RAG search | Yes |
| **llama-nemotron-rerank-1b-v2** | Reranker | Python | Better RAG ordering | Optional |
| **NanoOWL** (NVIDIA-AI-IOT) | Open-vocabulary detector (TensorRT) | Jetson AI Lab tutorial / container (jetson-containers works on JetPack 6.2.1) | Pre-label boxes for detection datasets | Yes |
| Nemotron Nano 9B v2 (4-bit) | LLM | llama.cpp | Optional quality comparison with 4B | About 6 GB: run alone |
| Cosmos Reason 2 8B (4-bit only) | Vision-language | llama.cpp | Optional | Default build listed at about 18 GB, so 4-bit only, with everything else stopped |
| Nemotron 3 Nano 30B-A3B | LLM | — | — | Too big: AGX Orin or a cloud GPU |

### Run Nemotron 3 Nano 4B (Build llama.cpp with CUDA for Orin = Compute Capability 8.7)

```bash
cd ~ && git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87
cmake --build build --config Release -j6          # 6 of your 8 cores; leaves RAM for the compiler
# Download NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf from the link on its Jetson AI Lab model page into ~/sutraai/models/
./build/bin/llama-server -m ~/sutraai/models/NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf \
  --alias nemotron --n-gpu-layers 999 --ctx-size 8192 --host 127.0.0.1 --port 8080
```

### Run Cosmos Reason 2 2B (Same llama.cpp Build; Can Run Next to Nemotron on 16 GB)

```bash
./build/bin/llama-server -hf Kbenkhaled/Cosmos-Reason2-2B-GGUF:Q8_0 -c 8192 --host 127.0.0.1 --port 8081
```
(This is the GGUF the Jetson AI Lab page points to for smaller Jetsons. Send images through the same OpenAI-style `/v1/chat/completions` API.)

**NanoOWL:** follow the Jetson AI Lab NanoOWL tutorial. It builds a TensorRT engine for the OWL-ViT model and gives a Python API you can call from a notebook.

---

## 9. Memory Budget for 16 GB

Approximate numbers; measure your own with jtop.

| Thing running | Roughly |
|---|---|
| Ubuntu + SSH + JupyterLab (desktop adds ~0.5–1 GB) | ~1–2 GB |
| PyTorch kernel training a MobileNet / 1D-CNN | ~1.5–4 GB |
| Nemotron 3 Nano 4B, Q4, 8k context | ~3.5–4 GB |
| Cosmos Reason 2 2B, Q8 | ~3 GB |
| Nemotron 1B embedder (fp16) | ~2.5 GB |
| Nemotron Nano 9B v2, Q4 (optional) | ~6 GB |

**Good combinations on 16 GB:** training + Nemotron 4B, or Nemotron 4B + embedder + Cosmos 2B. Avoid two big models plus training at once.

---

## 10. The Adoption Gate (Before Any Web Code)

Move to FastAPI / Next.js in Week 4 **only when all of these are ticked:**

- [ ] NB00 is green: GPU, cuDNN, TensorRT and ONNX Runtime CUDA are all visible
- [ ] **Image** (T5), **sensor** (T4/T6), **audio** (T1) and **detection** (T2) each go train → evaluate → export → int8 check
- [ ] Every int8 model: < 3-point accuracy drop (detection: < 0.05 F1)
- [ ] Each runs from **`pipeline.yaml` only**, via NB90, from a clean kernel
- [ ] Results reproduce within about 1% on a second run
- [ ] MFCC golden vectors saved; profiler table saved
- [ ] Device simulator: v1 → v2 OTA, sha256 check, telemetry and review list all work
- [ ] Nemotron advisor explains `metrics.json` correctly; the RAG notebook hits ≥ 8/10
- [ ] Reusable code lives in `sutra_core/*.py`, not only in notebook cells
- [ ] BUILDLOG has the numbers for every step

---

## 11. Courses: Udemy and Free

Udemy prices swing a lot, so buy during a sale. Before buying, check the **last-updated date** (2025 or later) and recent reviews.

### Udemy (Paid)

| # | Course | Why you need it | When |
|---|---|---|---|
| 1 | **PyTorch for Deep Learning Bootcamp (Zero to Mastery, Daniel Bourke)**. Also sold directly by ZTM: [zerotomastery.io/courses/learn-pytorch](https://zerotomastery.io/courses/learn-pytorch/) | The best beginner path for this project: tensors, training loop, computer vision, custom datasets, transfer learning. Free companion book: [learnpytorch.io](https://www.learnpytorch.io) | Weeks 1–2, daily |
| 2 | **LLM Engineering: Master AI, Large Language Models & Agents (Ed Donner)**: [udemy.com/course/llm-engineering-master-ai-and-large-language-models](https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models/) | RAG, QLoRA fine-tuning, agents. It matches your AI-engineer goal and the assistant feature. | Week 3 evenings → Month 2 |
| 3 | **Learn TensorFlow, PyTorch, TensorRT, ONNX From Scratch**: [udemy.com/course/learn-tensorflow-pytorch-tensorrt-onnx-from-scratch](https://www.udemy.com/course/learn-tensorflow-pytorch-tensorrt-onnx-from-scratch/) | Exporting models and TensorRT, the NB12 / NB42 skills. Check reviews first. | Week 1 (day 6) and Week 3 |
| 4 | **FastAPI – The Complete Course (Eric Roby)**. Course code: [github.com/codingwithroby/FastAPI-The-Complete-Course](https://github.com/codingwithroby/FastAPI-The-Complete-Course) | Week 4 move from notebooks to an API. You know web, so skim the basics. | Week 4 |
| 5 | Optional: **Practical Guide to Edge AI and TinyML for Smarter Devices**: [udemy.com/course/practical-guide-to-edge-ai-and-tinyml-for-smarter-devices](https://www.udemy.com/course/practical-guide-to-edge-ai-and-tinyml-for-smarter-devices/) | TinyML concepts for the ESP32 / Silicon Labs side. Check reviews first. | Month 2–3 |
| 6 | Optional: **The Complete Agentic AI Engineering Course (Ed Donner)** | Agents, for later assistant features | Month 2+ |

### Free (Do These Too)

| Resource | Why | When |
|---|---|---|
| **NVIDIA DLI: Getting Started With AI on Jetson Nano** (free, about 8 h; listed on [NVIDIA AI Learning Essentials](https://www.nvidia.com/en-sg/learn/ai-learning-essentials/)) | NVIDIA's own image-classification walkthrough on Jetson. It was written for the older Nano, but the concepts are the same. | Week 1 |
| **NVIDIA DLI: An Even Easier Introduction to CUDA** (free, 1 h) | What CUDA actually does, which helps you debug | Week 1 |
| **NVIDIA DLI: Building RAG Agents With LLMs** (paid, about US$90, optional) | Structured RAG, from NVIDIA | Month 2 |
| **[Jetson AI Lab](https://www.jetson-ai-lab.com)**: model pages + tutorials (Nemotron, Cosmos Reason, NanoOWL, Unsloth) | Exact commands for your board | Week 3 |
| **The Sound of AI** (YouTube, Valerio Velardo): audio signal processing for ML | The clearest MFCC / spectrogram explanations | Week 2 (NB30–NB32) |
| **NVIDIA forum guide: JetPack 7.2 on Orin Nano Super** ([link](https://forums.developer.nvidia.com/t/setting-up-the-nvidia-jetson-orin-nano-super-dev-kit-on-jetpack-7-2-a-practical-guide-june-2026/372490)) | Only if you're on JetPack 7.2 | Day 1–2 |

**Daily study rhythm:** 90 min course → 4 h notebooks → 30 min BUILDLOG.

---

## 12. Troubleshooting

| Symptom | Likely cause | Fix |
|---|---|---|
| `ImportError: libcudss.so.0` or `libcusparseLt.so.0` | JetPack 6.2.1 doesn't ship cuDSS / cuSPARSELt, which the torch 2.8.0 wheel needs | Install both from NVIDIA's Jetson downloads into `/opt/nvidia`, add `/etc/ld.so.conf.d` entries, `sudo ldconfig` (§5) |
| `ImportError: libnccl.so.2` or `undefined symbol: ncclCommWindowDeregister` | Early copy of the torch 2.8.0 wheel (since replaced on the index) | `pip install --force-reinstall --no-cache-dir --no-deps torch==2.8.0 --index-url https://pypi.jetson-ai-lab.io/jp6/cu126/+simple/` |
| Traceback shows `/usr/lib/python3/dist-packages` (`'RcParams' … '_get'`, `Unable to import Axes3D`, `'docstring'`) | The venv can see Ubuntu's old Python packages (made with `--system-site-packages`) | Switch to an isolated venv: set `include-system-site-packages = false` in `.venv/pyvenv.cfg`, link TensorRT, re-run the install lines (§5); restart JupyterLab and the kernel |
| `You are sending unauthenticated requests to the HF Hub` | Models download from Hugging Face without a login (harmless) | Create a free Read token at huggingface.co/settings/tokens, run `hf auth login` in the venv, restart the kernel |
| `torch.cuda.is_available()` is False | A CPU-only wheel came from normal PyPI | Uninstall torch/torchvision; reinstall from the Jetson index with `--no-cache-dir`; verify again |
| `RuntimeError: operator torchvision::nms does not exist` | torch and torchvision versions don't match | Install both from the same Jetson index, in one command |
| NumPy errors after installing something | NumPy 2 pulled in, or the version lock isn't active | `echo $PIP_CONSTRAINT` must print constraints-jp621.txt; reactivate the venv, then `pip install numpy==1.26.4` |
| Kernel dies / "Killed" during training | Out of memory | Smaller batch size, close other kernels, headless mode, check swap is on, stop any LLM |
| Very slow training | Low power mode or a data-loader bottleneck | `sudo nvpmodel -q` (MAXN SUPER?), `num_workers=2–4`, mixed precision |
| TensorRT not importable in the venv | The TensorRT link is missing | Run the `TRT_DIR=… / ln -sfn` lines in §5 |
| TFLite conversion fails | Python too old for onnx2tf 2.x (needs 3.12+), or an unsupported op | Convert in Google Colab with onnx2tf 2.6.9; simplify the model (MobileNetV2 / 1D-CNN) |
| llama.cpp is slow or uses the CPU | Built without CUDA or for the wrong GPU arch | Rebuild with `-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87` |
| A container fails on JetPack 7.2 (CUDA error) | Image built for CUDA 12.6 on a CUDA 13.2 driver | Use a CUDA 13 image, or build the tool from source |
| Board throttles or resets under load | Weak power supply or heat | Use the original PSU; check the fan; watch temperature in jtop |

---

## 13. Glossary for Beginners

| Term | Meaning |
|---|---|
| **Tensor** | A multi-dimensional array (like a NumPy array) that can live on the GPU |
| **Dataset split** | **Train** (model learns), **validation** (you tune), **test** (final honest score, touched once) |
| **Epoch / batch** | One pass over the training data / a small group of samples processed together |
| **Loss** | A number measuring how wrong the model is. Training pushes it down. |
| **Overfitting** | The model memorises training data: training accuracy up, validation accuracy down |
| **Transfer learning** | Start from a model pre-trained on millions of images and teach it your classes |
| **Augmentation** | Random changes (crop, flip, noise) that make the model robust |
| **Confusion matrix** | Table of true vs predicted classes; shows which classes get mixed up |
| **Precision / recall / F1** | Of my detections, how many were right? / Of the real objects, how many did I find? / The balance of both |
| **Window / stride** | Length of each time slice (e.g., 1 s) / how far the next slice moves (e.g., 0.5 s) |
| **MFCC** | Compact "fingerprint" of a sound slice: 13 numbers per 20 ms frame |
| **ONNX** | A standard file format for trained models that many runtimes can load |
| **TensorRT engine** | A model optimized for one specific NVIDIA GPU + TensorRT version |
| **TFLite / LiteRT** | Google's format and runtime for small devices. TFLite Micro runs on MCUs. |
| **Quantization (int8)** | Storing numbers as 8-bit integers instead of 32-bit floats: about 4× smaller and faster, slightly less accurate |
| **Calibration data** | Real samples used to choose the int8 scales during quantization |
| **Tensor arena** | The RAM block an MCU reserves for the model's working data |
| **Inference / latency** | Using the model to predict / how long one prediction takes |
| **GGUF** | File format for quantized LLMs, used by llama.cpp |
| **RAG** | Retrieval-Augmented Generation: find relevant text first, then let the LLM answer from it |
| **Embedding** | A vector of numbers that captures meaning; similar text gives nearby vectors |

---

## Sources

- JetPack 7.2.1 components: [NVIDIA JetPack archive 7.2.1](https://developer.nvidia.com/embedded/jetpack/downloads/archive-7.2.1)
- JetPack 7.2 Orin Nano setup guide (NVIDIA forum): [link](https://forums.developer.nvidia.com/t/setting-up-the-nvidia-jetson-orin-nano-super-dev-kit-on-jetpack-7-2-a-practical-guide-june-2026/372490)
- JetPack 6.2.2 for Orin: [JetsonHacks](https://jetsonhacks.com/2026/02/06/jetpack-6-2-2-for-jetson-orin/amp/)
- PyTorch for JetPack 6.2 (NVIDIA forum): [link](https://forums.developer.nvidia.com/t/pytorch-and-torchvision-for-jetpack-6-2/325257) · Jetson AI Lab package index: [pypi.jetson-ai-lab.io](https://pypi.jetson-ai-lab.io/)
- Jetson AI Lab: [Nemotron3 Nano 4B](https://www.jetson-ai-lab.com/models/nemotron3-nano-4b) · [Cosmos Reason 2 2B](https://www.jetson-ai-lab.com/models/cosmos-reason2-2b) · [NanoOWL tutorial](https://www.jetson-ai-lab.com/tutorials/nanoowl) · [Unsloth on Jetson](https://www.jetson-ai-lab.com/tutorials/unsloth-on-jetson) · [models list](https://jetson-ai-lab.com/models)
- NVIDIA Nemotron RAG models: [rerank 1B v2](https://docs.nvidia.com/nemo/automodel/model-coverage/reranking-models/nvidia/llama-nemotron-rerank-1b-v2) · [embed 1B v2](https://docs.api.nvidia.com/nim/re/reference/nvidia-llama-nemotron-embed-1b-v2)
- Courses: [NVIDIA AI Learning Essentials](https://www.nvidia.com/en-sg/learn/ai-learning-essentials/) · [ZTM PyTorch](https://zerotomastery.io/courses/learn-pytorch/) · [Best Udemy AI courses (travis.media)](https://travis.media/blog/best-udemy-ai-courses/)