Continuous Batching and KV Cache

Understand how vLLM-style schedulers interleave requests, reuse attention state, and trade throughput for latency.

In this module

• Why LLM Serving Is Different
• Continuous Batching
• KV Cache and the CPU-Safe vLLM Simulation