docs/research.md

# Research context

InferScale-Sim is a portfolio-scale implementation situated within the research line on replacing expensive deployment sweeps with modeling, simulation, or emulation.

## Vidur (2024)

Vidur combines experimental profiling, predictive models, and end-to-end inference simulation. Its paper reports under 9% inference-latency estimation error across the evaluated range and a configuration-search example where LLaMA2-70B exploration took about one CPU-hour versus an estimated 42K GPU-hours (~$218K) for deployment-based exploration.

InferScale inspiration: workload-aware configuration search, explicit performance-profile provenance, and separation between latency prediction and event simulation.

Source: https://arxiv.org/abs/2405.05465

## SGLang / RadixAttention (2024)

SGLang introduced RadixAttention for automatic KV-cache reuse across shared prompt prefixes. A radix tree retains reusable prompt/generation state and combines prefix lookup with cache-aware scheduling and eviction.

InferScale v0.3 does not implement a radix tree. It adds a deliberately smaller controlled shared-prefix abstraction so the effect of avoiding repeated prefill and duplicate KV allocation can be studied independently.

Source: https://arxiv.org/abs/2312.07104

## TokenSim (2025)

TokenSim emphasizes extensible exploration of scheduling and memory-management choices and reports sub-1% error in its evaluated workloads.

InferScale inspiration: modular scheduler/memory experiments.

Source: https://arxiv.org/abs/2503.08415

## Revati (2026)

Revati avoids reimplementing every evolving serving-system control decision: it executes real vLLM/SGLang control paths while virtualizing CUDA time. The paper reports less than 5% prediction error across its evaluated scenarios and 5-17x faster execution than real GPU serving runs.

InferScale lesson: pure simulation has a control-path fidelity ceiling. Our schedulers therefore document their semantics rather than claiming framework equivalence.

Source: https://arxiv.org/abs/2601.00397

## LLMServingSim 2.0 (2026)

LLMServingSim 2.0 focuses on heterogeneous and disaggregated infrastructure and integrates batching, routing, offloading, memory, communication, and power into a unified runtime loop. The 2026 paper reports about 0.97% average error in its validation.

InferScale v0.3 inspiration: role-specific resources and explicit data movement rather than assuming a monolithic replica.

Source: https://arxiv.org/abs/2602.23036

## Frontier (May 2026)

Frontier models co-location, Prefill-Decode Disaggregation, Attention-FFN Disaggregation, runtime optimizations such as speculative decoding, and stateful workloads. It reports average throughput error below 4% on its 16-H800 evaluation and explicitly supports SLA-dependent Pareto-frontier exploration.

InferScale v0.3 inspiration:

- separate prefill and decode worker pools;
- communication/KV-transfer cost;
- role-specific bottleneck telemetry;
- an interactive bounded Pareto design-space study.

Source: https://arxiv.org/abs/2605.21312

## Scope boundary

InferScale-Sim v0.3 is not intended to compete with these research systems on fidelity, hardware scale, or runtime compatibility. Its goal is an inspectable, dependency-light Python implementation with a zero-backend interactive interface for understanding the systems mechanisms that motivate them.


## HeteroPanacea / When Does Disaggregation Pay? (Aug 2026)

**When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference** pushes the simulator design space toward heterogeneous stage-specific hardware and automated cross-stack exploration. InferScale-Sim v0.3 does not model Attention--FFN disaggregation or custom NPUs, but its P/D topology already permits independent prefill and decode accelerator profiles and reports resource-normalized efficiency so heterogeneous layouts can be compared without treating extra accelerators as free.

- arXiv: https://arxiv.org/abs/2608.03741