Topic
LLM Serving
Running the model yourself — vLLM, batching, throughput and latency budgets, and the arithmetic behind the surprises.
1 post · ← all posts
Temperature zero is not deterministic
Same prompt, same model, different answer. The mechanism behind non-determinism, the arithmetic that kills agent pilots, and what to do about it.