ReferenceUse alongside every step
Courses, NVIDIA models and glossary
What to study each week, which NVIDIA models fit your 16 GB Orin NX, how memory gets shared, and plain-English meanings for the terms you'll meet.
Study plan by week
| When | Course (90 min a day) | Pairs with |
|---|---|---|
| Week 1 | NVIDIA DLI "Getting Started With AI on Jetson Nano" + "An Even Easier Introduction to CUDA" (both free) · ZTM PyTorch: fundamentals, workflow, computer vision | Steps 01–05 |
| Week 2 | ZTM PyTorch: custom datasets, transfer learning · The Sound of AI (YouTube) on spectrograms and MFCC | Steps 06–08 |
| Week 3 | TensorRT/ONNX course · Ed Donner LLM Engineering: RAG section · Jetson AI Lab tutorials | Steps 09–13 |
| Week 4 | FastAPI – The Complete Course (skim; you already know web) | Steps 14–16 |
Udemy courses
Udemy prices swing a lot, so buy during a sale. Before buying, check the last-updated date (2025 or later) and recent reviews.
| Course | Why | When |
|---|---|---|
| PyTorch for Deep Learning Bootcamp (Zero to Mastery, Daniel Bourke). Course page; also on Udemy. Free book: learnpytorch.io | The best beginner path for this project: tensors, training loop, vision, custom data, transfer learning | Weeks 1–2 |
| LLM Engineering: Master AI, Large Language Models & Agents (Ed Donner). Udemy | RAG, QLoRA fine-tuning, agents: your AI-engineer track and the SutraAI assistant | Week 3 → Month 2 |
| Learn TensorFlow, PyTorch, TensorRT, ONNX From Scratch. Udemy | Model export and TensorRT (Steps 05 and 09). Check reviews first. | Weeks 1 and 3 |
| FastAPI – The Complete Course (Eric Roby). Course code on GitHub | Step 15 | Week 4 |
| Optional: Practical Guide to Edge AI and TinyML for Smarter Devices. Udemy | TinyML ideas for the ESP32 / Silicon Labs side. Check reviews first. | Months 2–3 |
| Optional: The Complete Agentic AI Engineering Course (Ed Donner) | Agents, for later assistant features | Month 2+ |
Free resources
- NVIDIA DLI on AI Learning Essentials: "Getting Started With AI on Jetson Nano" (free, about 8 h), "An Even Easier Introduction to CUDA" (free, 1 h), "Building RAG Agents for LLMs" (free, 8 h) — see the free courses page
- Jetson AI Lab: model pages and tutorials with exact commands for your board
- The Sound of AI on YouTube (Valerio Velardo): audio signal processing for ML
- NVIDIA: JetPack 6.2 Super Mode for Orin NX: power modes, MAXN SUPER, thermals
NVIDIA models to test on your Orin NX 16 GB
| Model | Type | Run with | Use in SutraAI | Step |
|---|---|---|---|---|
| Nemotron 3 Nano 4B (Q4_K_M GGUF) | LLM | llama.cpp llama-server | Training Advisor, RAG answers, in-app assistant | 11 |
| Cosmos Reason 2 2B (Q8_0 GGUF) | Vision-language | llama.cpp | Check labels, describe images, flag odd samples | 12 |
| llama-nemotron-embed-1b-v2 | Embeddings | Python (Hugging Face) | RAG search | 11 |
| llama-nemotron-rerank-1b-v2 | Reranker | Python | Better RAG ordering (optional) | 11 |
| NanoOWL | Open-vocabulary detector | TensorRT, Jetson AI Lab tutorial | First-draft boxes for labelling | 12 |
| Nemotron Nano 9B v2 (4-bit GGUF) | LLM | llama.cpp | Optional quality comparison with the 4B model. Run alone, about 6 GB. | 11 |
| Cosmos Reason 2 8B (4-bit GGUF only) | Vision-language | llama.cpp | Optional. Its default build is listed at about 18 GB, so use 4-bit only, with everything else stopped. | 12 |
| Nemotron 3 Nano 30B-A3B | LLM | — | Too big for 16 GB alongside anything else: AGX Orin or a cloud GPU | — |
Memory budget (16 GB shared by CPU and GPU)
| Running | Roughly |
|---|---|
| Ubuntu + SSH + JupyterLab (desktop adds about 0.5–1 GB) | 1–2 GB |
| PyTorch kernel training MobileNet or a 1D-CNN | 1.5–4 GB |
| Nemotron 3 Nano 4B, Q4, 8k context | 3.5–4 GB |
| Cosmos Reason 2 2B, Q8 | about 3 GB |
| Nemotron 1B embedder (fp16) | about 2.5 GB |
| Nemotron Nano 9B v2, Q4 (optional) | about 6 GB |
These are approximate; measure yours with jtop. Good combinations on 16 GB: training + Nemotron 4B, or Nemotron 4B + embedder + Cosmos 2B. Avoid two big models plus training at once.
Common problems
| Symptom | Cause | Fix |
|---|---|---|
ImportError: libcudss.so.0 / libcusparseLt.so.0 | JetPack 6.2.1 doesn't ship cuDSS / cuSPARSELt, which torch 2.8.0 needs | Install both into /opt/nvidia and run sudo ldconfig (Step 02, substep 3) |
libnccl.so.2 / undefined symbol: ncclCommWindowDeregister | Early copy of the torch 2.8.0 wheel | Force-reinstall torch 2.8.0 from the jp6/cu126 index with --no-cache-dir --no-deps |
torch.cuda.is_available() is False | CPU wheel from PyPI | Reinstall torch from the Jetson index with --no-cache-dir (Step 02) |
Traceback shows /usr/lib/python3/dist-packages ('RcParams' … '_get', Unable to import Axes3D, 'docstring') | The venv can see Ubuntu's old Python packages (made with --system-site-packages) | One-time switch to an isolated venv with only TensorRT linked (Step 02, substep 2); restart JupyterLab and the kernel |
You are sending unauthenticated requests to the HF Hub | Downloading models from Hugging Face without logging in (harmless) | Free Read token + hf auth login in the venv (Step 02, substep 5) |
torchvision::nms does not exist | torch/torchvision mismatch | Install torch 2.8.0 + torchvision 0.23.0 together from the jp6/cu126 index |
| torch version changed after installing a package | Version lock not active | echo $PIP_CONSTRAINT should print constraints-jp621.txt; reactivate the venv (Step 02) |
| onnx2tf install compiles for ages or breaks numpy | Old onnx2tf 1.29 picked for Python 3.10, or installed into the main venv | onnx2tf 2.6.9 in Colab or in a Python 3.12 .venv-convert (Step 05, substep 4) |
| Kernel "Killed" | Out of memory | Smaller batch, close kernels, headless, swap on, stop LLMs |
| Slow training | Power mode or data loading | MAXN SUPER (sudo nvpmodel -m 0), num_workers=4, mixed precision |
| TFLite conversion fails | Python too old for onnx2tf 2.x, or an unsupported op | Convert in Colab (Step 05, Option A); prefer MobileNetV2 / 1D-CNN |
| llama.cpp on the CPU | Built without CUDA | Rebuild with -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87 |
| Container CUDA error on JetPack 7.2 | CUDA 12.6 image vs 13.2 driver | CUDA 13 image or build from source |
Glossary
| Term | Meaning |
|---|---|
| Tensor | A multi-dimensional array that can live on the GPU |
| Train / validation / test | Learn from it / tune against it / final honest score, used once |
| Epoch / batch | One pass over the training data / a small group processed together |
| Loss | How wrong the model is; training pushes it down |
| Overfitting | Memorising the training data: training score up, validation score down |
| Transfer learning | Start from a model pre-trained on millions of images and teach it your classes |
| Augmentation | Random changes (crop, colour, noise) that make the model robust |
| Confusion matrix | True vs predicted classes; shows which classes get mixed up |
| Precision / recall / F1 | How many detections were right / how many real objects were found / the balance of both |
| Window / stride | Length of each time slice / how far the next slice moves |
| MFCC | A compact fingerprint of a sound slice: 13 numbers per 20 ms frame |
| ONNX | A standard model file format many runtimes can load |
| TensorRT engine | A model optimized for one specific NVIDIA GPU + TensorRT version |
| TFLite / LiteRT | Google's model format and runtime for small devices; TFLite Micro runs on MCUs |
| Quantization (int8) | 8-bit integers instead of 32-bit floats: about 4× smaller and faster, slightly less accurate |
| Calibration data | Real samples used to pick the int8 value ranges |
| Tensor arena | The RAM block an MCU reserves for the model's working data |
| Golden vectors | Saved input/output pairs that a new implementation (e.g. C on a chip) must reproduce exactly |
| GGUF | File format for quantized LLMs used by llama.cpp |
| RAG | Find relevant text first, then let the LLM answer only from it |
| Embedding | A vector that captures meaning; similar text gives nearby vectors |