Prefill, decode, the KV cache, paged attention, continuous batching, speculative decoding and the three kinds of parallelism, mapped onto the search-cluster concepts you already know, plus the places where the analogy breaks.
Prefill, decode, the KV cache, paged attention, continuous batching, speculative decoding and the three kinds of parallelism, mapped onto the search-cluster concepts you already know, plus the places where the analogy breaks.