Writing
Field notes
What breaks when an LLM system meets production, and the order to fix it in. RSS.
-
The half of your GenAI system nobody put on call
Nobody's RAG system breaks — it quietly stops being right. The data jobs nobody wrote down, the three mechanisms that make them safe, and what to put on a dashboard.
-
Temperature Zero Is Not Deterministic
Same prompt, same model, different answer. The mechanism behind non-determinism at temperature zero, the arithmetic that kills agent pilots, and what to do about it.
-
Your RAG doesn't have a retrieval problem. It has a ranking problem.
The document was in the index and the answer was still wrong. How to tell a retrieval failure from a ranking failure, in what order to fix them, and what each fix costs.