InferScale-Sim / docs /validation.md
ArchitSharma's picture
Commiting v0.3
44745f2
|
Raw
History Blame Contribute Delete
1.88 kB
# v0.3 validation checklist
The release checks prevent software, deployment, and provenance mistakes; they do not pretend the analytical profile has measured-hardware fidelity.
## Automated
- deterministic workload generation
- constant/Poisson/bursty workload behavior
- prefill latency monotonicity
- quantization footprint ordering
- colocated end-to-end completion
- static vs continuous behavioral difference
- component TTFT/E2E SLO accounting
- robust repetition-aware capacity search
- bottleneck-diagnosis provenance
- prefix hits do not alter the underlying generated request trace
- prefix reuse reduces modeled prefill work
- P/D pipeline completion
- non-zero P/D transfer telemetry
- configurable P/D worker counts
- four-scenario topology/cache comparison
- bounded design-space sweep and Pareto marking
- ASCII-only public UI labels
- chart export controls present
- explicit planner worst-repetition and target columns
- no product-style footer
- Hugging Face `short_description` <= 60 characters
- `sdk: static` metadata
- canonical Python source equals browser mirror
- every Python module is included by the worker
- provenance remains `analytical-reference`
- JavaScript syntax parse
- Python compilation
## Not claimed in v0.3
- empirical L4/A10G/A100 latency accuracy
- exact vLLM/SGLang scheduler equivalence
- CUDA-kernel modeling
- real network-protocol fidelity
- radix-tree prefix-cache eviction/scheduling
- speculative decoding
- Attention-FFN disaggregation
- multi-turn/agentic session fidelity
These remain explicit future extensions.
## Design-space sanity checks
The release suite verifies that the bounded design sweep returns at least one raw-performance Pareto point and at least one resource-efficiency Pareto point. P/D candidates also expose accelerator-instance counts so raw and normalized throughput can be interpreted separately.