docs/FormScout-Starter-Kit.md

FormScout — Starter Kit & Resource Pack

Companion to FormScout-FMS-Spec.md and FormScout-Build-Prompt.md. Every link below was checked. Read §1 first — some items are time-sensitive and block the build if you leave them late.

---

1. Do this NOW (before the hack window — some take hours to clear)

• Request access to the gated Meta checkpoints today. Both are gated on Hugging Face and approval isn't instant:
  • SAM 3 / SAM 3.1 — request on the SAM 3 repos (you need the latest code for the 3.1 checkpoints).
  • SAM 3D Body — facebook/sam-3d-body-dinov3 and facebook/sam-3d-body-vith both require an access request, then an authenticated download. Note: data/checkpoints are blocked in sanctioned jurisdictions — shouldn't affect SK, but verify.
• Put your HF token in the Space secrets so the Space can pull the gated weights at build time.
• Check licenses before you commit to a model (this affects whether you can even submit):
  • Qwen3-VL-8B / Qwen3-VL-Embedding-8B / Qwen3.6 → Apache-2.0 (clean).
  • SAM 3 / SAM 3.1 / SAM 3D Body → SAM License (not Apache; read the terms — there are use restrictions).
  • Ultralytics YOLO26 → historically AGPL-3.0 (open-sourcing obligations; commercial license exists). Verify on the model/repo and make sure an AGPL dependency is OK for your submission. If it's a problem, RTMPose/ViTPose are alternatives.
  • pyskl / MMAction2 → Apache-2.0.
  • KIMORE / UI-PRMD → academic/research terms; check before redistributing anything derived.
• Confirm the param-counting rule in the Discord AMA. Specifically: (a) is it summed across the pipeline or per-model? (b) do frozen base models count? (c) does a LoRA adapter's base count? Your ~18B config is safe under the strict reading either way, but get it on record.

---

2. Literature package

2.1 The framing that wins — "evaluate like an FMS reliability study"

The single most credible move in your writeup: evaluate FormScout the way the clinical literature evaluates human FMS raters. Treat the model as a second rater and report weighted Cohen's κ and ICC against the physio, the exact metrics the reliability papers use. That instantly makes your results legible to any sports-medicine reader and is far more honest than a vanity accuracy number.

Resource	What it gives you	Link
Physiopedia — FMS	Clean overview of the 7 tests + 0–21 scoring	https://www.physio-pedia.com/Functional_Movement_Screen_(FMS)
FMS reliability study (JOSPT 2012)	The ICC/κ numbers and method you'll mirror in your eval	https://www.jospt.org/doi/10.2519/jospt.2012.3838
FMS in elite youth soccer (PMC)	Per-test scores, asymmetries, clearing-test order	https://pmc.ncbi.nlm.nih.gov/articles/PMC5675373/
Clinician's guide to FMS scoring	Per-test 3/2/1 criteria in plain language (rubric source)	https://meloqdevices.com/blogs/meloq-updates/functional-movement-screening

Honesty anchor for the blog post: the popular "≤14 → injury risk" cutoff has weak/mixed predictive validity. Sell standardization, asymmetry detection, and a repeatable baseline — not prediction.

2.2 Action Quality Assessment — surveys & living lists

Resource	Why	Link
A Decade of AQA (survey, 2025, 200+ papers, PRISMA)	The map of the whole field; start here	https://arxiv.org/abs/2502.02817 · code: https://github.com/HaoYin116/Survey_of_AQA
Comprehensive Survey of AQA: Method & Benchmark (2024)	Taxonomy by modality (video / skeleton / multimodal) + unified benchmark	https://arxiv.org/abs/2412.11149 · page: https://zhoukanglei.github.io/AQA-Survey
Awesome-AQA (ZhouKanglei)	Curated, has a Medical-Care/rehab section — your closest analogues	https://github.com/ZhouKanglei/Awesome-AQA
Awesome-AQA (Lyman-Smoker)	Second list; catches papers the other misses (FLEX, ExAct, etc.)	https://github.com/Lyman-Smoker/Awesome-AQA

2.3 Skeleton-based scoring — the methods your head will borrow from

Paper	Relevance to FormScout	Link
ST-GCN (original)	The graph-over-skeleton + temporal-conv backbone	https://github.com/open-mmlab/mmaction2/blob/main/configs/skeleton/stgcn/README.md
AQA via Hierarchical Pose-guided Multi-Stage Contrastive Regression (TIP 2025)	Pose-guided + contrastive regression with few labels — close to your setup	https://arxiv.org/abs/2501.03674
Attention-guided Movement Quality Assessment + skeletal augmentation (UI-PRMD/KIMORE)	Transformer MQA on clinician-scored rehab data; augmentation recipe for tiny sets	https://arxiv.org/pdf/2204.07840
SSL-Rehab: self-supervised 3D skeleton + LoRA fine-tune (KIMORE/UI-PRMD)	Pretrain→LoRA recipe for small clinical datasets (uses your LoRA muscle)	https://www.sciencedirect.com/science/article/abs/pii/S1077314224003564
Skeleton-based AQA w/ anomaly-aware DTW (Sensors 2025)	DTW alignment + anomaly scoring; cheap, label-light baseline	https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12693942/

---

3. Models & tooling (verified)

Component	Repo / card	Params	License	Gated?
YOLO26-Pose	https://docs.ultralytics.com/tasks/pose	<0.1B	AGPL-3.0*	no
SAM 3.1	https://github.com/facebookresearch/sam3	~0.85B	SAM License	yes
SAM 3D Body	https://github.com/facebookresearch/sam-3d-body · https://huggingface.co/facebook/sam-3d-body-dinov3	sub-1B†	SAM License	yes
ST-GCN++ / PoseConv3D	https://github.com/kennymckormick/pyskl	~0.01–0.05B	Apache-2.0	no
Qwen3-VL-8B-Instruct	https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct	8B	Apache-2.0	no
Qwen3-VL-Embedding-8B	https://huggingface.co/Qwen/Qwen3-VL-Embedding-8B (GGUF: dam2452/...-GGUF)	8B	Apache-2.0	no
Qwen3.6-27B (alt brain)	https://huggingface.co/unsloth/Qwen3.6-27B-GGUF	27B	Apache-2.0	no

* verify the current YOLO26 license. † two variants (dinov3, vith); confirm exact count on the card — budget impact is small either way. SAM 3 itself is 848M.

Useful extras: SAM 3D Body uses a Momentum Human Rig (MHR) that separates skeleton from soft-tissue shape — convenient for clean joint-angle extraction. The repo ships a notebook combining SAM 3D Body + SAM 3D Objects in one frame of reference. SAM 3D Body demo: https://www.aidemos.meta.com/segment-anything/editor/convert-body-to-3d

---

4. Datasets for transfer / pretraining

You have a couple of labeled clips. Pretrain on clinician-scored movement-quality data first, then few-shot fine-tune. These are the most transferable to FMS (ranked by relevance):

Dataset	Why it's the closest analogue	Link
KIMORE	Clinician scores of low-back-pain rehab exercises (trunk control, multi-plane) — same "score movement quality" task as FMS; partially overlaps Deep Squat / Rotary Stability / TSPU mechanics	https://www.researchgate.net/publication/333791841 (search "KIMORE dataset")
UI-PRMD	10 rehab movements, correct vs. incorrect executions; standard MQA benchmark, pairs with KIMORE	search "UI-PRMD University of Idaho Physical Rehabilitation Movements"
Fitness-AQA	Real gym squat/deadlift form errors — directly relevant to Deep Squat compensations	https://github.com/ParitoshParmar/MTL-AQA (links Fitness-AQA)
FLEX	Large multi-modal fitness AQA dataset	via Lyman-Smoker/Awesome-AQA
MTL-AQA / AQA-7 / FineFS	General sports AQA for backbone pretraining (diving, skating)	https://github.com/ParitoshParmar/MTL-AQA

FMS-specific public video data is scarce — don't expect a drop-in set. Your physio's clips are the gold; everything above is for pretraining the temporal backbone so it learns movement structure before it ever sees an FMS label.

---

5. Build & deploy tooling

Need	Link
Gradio docs (v6)	https://www.gradio.app/docs
gradio.Server — custom frontend + Gradio backend (Off-Brand badge)	https://www.gradio.app/guides/server-mode · blog: https://huggingface.co/blog/introducing-gradio-server
Gradio AI coding-assistant skill	gradio skills add --claude (PyPI: https://pypi.org/project/gradio/)
Gradio changelog (confirm gr.Walkthrough, gr.Navbar, gr.Video.playback_position)	https://www.gradio.app/changelog
HF Spaces ZeroGPU (@spaces.GPU)	https://huggingface.co/docs/hub/spaces-zerogpu
llama.cpp	https://github.com/ggml-org/llama.cpp
pyskl (ST-GCN++/PoseConv3D, custom-video tutorial incl. diving48)	https://github.com/kennymckormick/pyskl
MMAction2 (broader video understanding)	https://github.com/open-mmlab/mmaction2
Hackathon's own trailheads (ML Intern, Gradio guides)	https://github.com/huggingface/ml-intern

Hackathon-specific gotcha already seen in the org: another team's Space hit libcudart.so.12 errors and had to swap llama.cpp for transformers + spaces.GPU. Plan for it — isolate the llama.cpp build (CPU-only or pinned-CUDA) and keep a transformers fallback. For the scoring head, a small hand-rolled ST-GCN may deploy more cleanly on a Space than the full MMAction2/pyskl stack — prototype with pyskl, ship lean.

---

6. Two artifacts you probably haven't made yet

6.1 Data & capture protocol (highest-leverage non-code work)

With a tiny dataset, controlling how clips are captured beats any model tweak. Give the physio a one-pager:

• Camera: one fixed position, tripod, ~3 m back, lens at hip height, landscape, 1080p/30fps+. Same setup every session — this is what makes 3D consistent and the longitudinal baseline meaningful.
• Framing: whole body in frame for the whole rep, including the dowel. Plain-ish background, even lighting, no backlight.
• One athlete in frame at scoring time (or note who to track). For bilateral tests, capture both sides and label each.
• Label schema (CSV): clip_id, athlete_id, date, test_name, side(L/R/NA), score(0–3), pain(bool), compensation_notes(free text), camera_view, consent_on_file(bool).
• One rep per clip to start (simplest). If sessions are continuous, you'll need temporal segmentation first — flag it to the build agent at Phase 1.

6.2 Evaluation plan

Define "good" before you train, given so few labels:

• Primary: Spearman ρ between predicted and physio scores (the AQA-standard metric), plus exact-match and ±1 accuracy per test.
• Clinical credibility: weighted Cohen's κ and ICC of model-vs-physio, reported alongside the human inter-rater numbers from the JOSPT study — i.e. "how does FormScout compare to a second human rater?"
• Asymmetry: detection rate of L/R asymmetries the physio flagged (this is one of the FMS's most defensible outputs).
• Validation: leave-one-clip-out CV (you can't afford a held-out test split). Keep ≥1 clip the judge never sees for the demo.
• Calibration: report when the system says "low confidence / physio review" and show it's right to do so. A well-calibrated, humble tool reads as more trustworthy than a confident one.

---

7. Ethics, consent & data handling (EU / Slovakia)

You're filming identifiable athletes, possibly minors on a youth team. This is biometric personal data under GDPR — treat it as first-class, and say so in your submission (judges and physios both reward it):

• Consent: written consent from each athlete (and a parent/guardian for anyone under 18) before any footage is used. No consent → not in the dataset, not in the demo.
• Data minimization & retention: keep only what you need; don't persist raw clips on the Space beyond what's approved; document a retention/deletion policy. Prefer storing derived skeletons over raw video where possible.
• Demo footage: use a consenting adult (you, a teammate) for the public demo video rather than a minor athlete, even if you trained on team data privately.
• Framing: screening aid, not a medical device; pain/clearing tests always defer to the clinician; human-in-the-loop by design.

---

8. The transfer-learning recipe (ties it together)

1. Backbone pretrain — ST-GCN++ on a general skeleton-action set (NTU/Kinetics skeletons via pyskl) so it learns motion structure.
2. Domain adapt — continue on KIMORE + UI-PRMD (clinician-scored movement quality) so it learns quality, not just what action.
3. Few-shot fine-tune — LoRA on the physio's FMS clips with heavy augmentation (temporal jitter, L↔R mirror to double bilateral data, 3D camera-angle perturbation, joint noise). The SSL-Rehab paper (§2.3) is your blueprint and it's exactly your LoRA wheelhouse.
4. Don't over-train the head — let deterministic biomechanics carry the demo; the learned head and RAG are the refinement and the badges, not the foundation.

---

9. Demo & submission storyboard (the "make it sing" 30%)

The submission needs a demo video + social post; "Show, Don't Tell" is a literal rule. A tight 60–90s cut:

1. 0–10s — the problem: physio eyeballing a squat, scribbling a score. "Same player, two raters, two scores."
2. 10–35s — upload the clip to FormScout → skeleton overlay → 0–3 with the deciding angle drawn on the frame (playback_position jump). The "aha" shot.
3. 35–55s — the scorecard: composite 0–21, the L/R asymmetry strip, a "low confidence — physio review" flag on a borderline case (honesty sells).
4. 55–75s — the physio reacting / using it on a real player (the Backyard AI "they actually used it" proof).
5. End card — "Runs on a laptop. ~18B params. Screening aid, not a diagnosis." Link the Space, the published head, the agent trace, the blog.

Social post: lead with the overlay GIF + the asymmetry-detection angle; tag Gradio/HF; one line of honest framing.

---

Built to give FormScout the best shot. The two things most teams underinvest in — the capture protocol (§6.1) and the honest, clinical-style evaluation (§6.2, §2.1) — are exactly where this project can out-class flashier entries. Good luck. 🏀