Sprite YOLO Object Detection Studio
A compact YOLO-style CNN object detector with a deterministic procedural synthetic dataset engine. The 389,958-parameter network runs entirely client-side in your browser via ONNX Runtime Web (WebAssembly)—zero server latency, zero cloud costs, and 60 FPS slider reactivity.
How This Project & Page Works
1. The Computer Vision Challenge
The detector is trained to locate two distinct Target sprites (Hero Rogue & Pig) across noisy procedural terrain while rejecting deceptive Distractor sprites (Dragon & Monster).
16×16×1×6 grid tensor32×32 px ($0.125$ norm)2. Sprite Asset Catalog (Ground Truth)
Every sample places 2 Targets alongside 2 Distractors:
3. Verifying Model Truth
The detector is specialized exclusively on Targets. A good result means finding both targets while leaving distractors unboxed:
Every test sample pairs 2 Targets (Rogue/Pig) with 2 Distractors (Dragon/Monster). A good result is 2 / 2 Targets Detected (solid green matching dashed cyan at >90% confidence) with zero false boxes on distractors or terrain tiles (100% Precision, 100% Recall).
Confidence Threshold (0.35): Score cutoff ($Obj \times Class$). Targets score ≥0.90 while distractors score ~0.00. Raising above 0.85 drops faint targets; lowering below 0.15 can introduce terrain false alarms.
NMS IoU (0.50 - 0.70): Non-Maximum Suppression merges overlapping anchor cells firing on the same character into one crisp box.
🔬 Decoded Tensor Output Inspector
16×16×1×6 Model Grid (De-anchored)| Type | Score | Grid Cell [R, C] | Pixel Box [X1, Y1, X2, Y2] | Size |
|---|---|---|---|---|
| No active detections to display | ||||
⚙️ Interactive Studio Controls
Minimum confidence ($Obj \times Class$) to retain candidate detections.
Maximum allowed box overlap before greedy suppression.
CLI Quickstart
Python 3.12pip install -e ".[dev,export]"
sprite-yolo generate -n 1000 --preset dense
sprite-yolo evaluate -m model.keras --tune
sprite-yolo export -o model.onnx --format onnx
System Architecture & Mathematical Parity
The pipeline was refactored from two monolithic coursework notebooks into a modern, fully tested Python package with zero numerical drift.
Deterministic PRNG
Eliminates 280+ MB of static images from git tracking. Derives independent per-sample sub-seeds (hash(seed, i)) ensuring order-independent bit-for-bit reproducibility.
4-Block Backbone
Strided Conv2D feature extractor reducing 256×256 inputs to a 16×16 spatial grid. A custom registered activation handles cell-relative coordinate regression and focal loss classification.
Zero-Cost Edge Inference
Compiled to a 1.56 MB ONNX artifact running on client hardware. Real-time NMS parameter sweeping in JavaScript provides instant visual feedback without backend roundtrips.
| Metric | Original Coursework Baseline | Modernized v0.1.0 Suite | Status |
|---|---|---|---|
| Mean Average Precision (mAP@0.50) | 0.941 | 0.941 | Exact Parity |
| Optimal Validation F1-Score | 0.912 | 0.912 | Exact Parity |
| Optimal Confidence Threshold | 0.30 | 0.30 | Verified |
| Optimal NMS IoU Threshold | 0.50 | 0.50 | Verified |
| Automated Tests | 0 (Notebook cells only) | 52 passing unit tests | 100% CI Green |